The Complete Guide to SEO Testing: Best Practices, Experiment Ideas, Tool Walkthrough

How do you prove the work you are doing to a website is offering a return on investment by getting better rankings and more clicks from Google?

One way is implementing a test-driven SEO process of:

Change -> Test -> Learn 

This allows you to pinpoint what is working, what isn’t working, and how the business should invest in it’s SEO strategy going forward.

In this guide we will define what SEO testing is and describe valuable use cases. We’ll cover best practices, how to track results, and cover an extensive list of ideas for SEO experiments you could run and why you should consider adding SEO testing to your SEO workflow.

Finally we’ll walk you through how to setup an SEO test using SEOTesting.com.

Table of contents

• What is SEO testing?
• Why run SEO tests?
• SEO audit vs SEO test
• 5 business cases for SEO testing
• SEO testing best practices
• Ideas for test-driven SEO experiments you can try
• How to track results

What Is SEO Testing?

SEO testing is the process of measuring and evaluating the results of a change you make to a page or your site in relation to organic search traffic.

We look at clicks, impressions, position, click through rate, and the number of queries ranked for. This can be based on an individual page or section of a site over two time periods, a specific query, two separate groups of pages you run as a split test, or a group of pages you continually track to help you show your work is bearing fruit. You can successfully incorporate SEO testing into your SEO checklist to ensure success.

This brings us nicely to why we should be running SEO Tests…

Why Run SEO Tests?

SEO testing shows you what actually works.

Google’s ranking algorithm is a black box. In fact there are multiple algorithms that are fed into other algorithms, that come together to produce an overall ranking of a page or website.

There are over 200 ranking factors the algorithms use, with machine learning working alongside these signals to try and show the best results it can for the user’s individual query. It has been reported that the Google search team are unsure of the weight of individual signals as machine learning has taken over judging which are the most important.

Different niches are also evaluated against different criteria and factors. Travel websites are likely to use different ranking factors to medical websites – and potentially completely different ranking algorithms.

Spend an hour on Twitter and you’ll get a hundred different opinions on what it takes to improve rankings in Google. All these opinions may have worked in the past – but are they still applicable, and relevant to a site in your niche?

Running a website change as an SEO test allows you to evaluate these ideas and see whether they work on your site and in the niche it sits.

Once you see a positive result you can confidently roll out the change to other pages. If the results aren’t positive you can roll the change back.

You can then move onto the next thing to optimize and test.

Systematic testing turns ad-hoc website changes into a methodical approach of collecting insight about what works for a particular website. When done long term, test-driven SEO tends to lead to a snowball effect, often visualized by exponential growth curves.

SEO Audit vs SEO Test

Sometimes there is a bit of confusion over what is an SEO audit and what is an SEO test.

An SEO audit is the process of looking at the current state of an existing site or page, and based on technical best practices and rule sets, coming up with a list of things to fix, improve or change.

SEO site audits are valuable because they review the overall state of your website. They may discover inconsistencies in your URI structure, pages that should be noindexed or have canonical URLs, images that are missing alt tags or have disproportionate file sizes. SEO audits can even discover content gap opportunities or information architecture flaws.

An SEO test is the process of tracking the results, since a change has been made. Quite often, you should look to do an audit, and run SEO tests on the fixes the audit suggests. This way you can evaluate the effectiveness of the recommended changes.

5 Business Cases For SEO Testing

SEO testing is the super power that can help internal SEOs and agencies demonstrate the value and return of their work. Building a test-driven SEO approach into your work practices allows you to have a continuous method of improvement to make sure your website does not stagnate.

We’ve outlined 5 business cases to help you propose a test-driven SEO approach to project owners.

#1 Find Out What Works, And Scale It Up

It can be costly to implement a change across an entire site, and it can also be risky if you do not know how the change will affect rankings, especially for new businesses with limited budgets just setting up their business websites.

By using an SEO testing approach you can make your change to a single page, or a group of pages, and track the results.

Upon getting a positive result, you can roll the change out across the site, or to a larger test group if you want to make sure the initial test results replicate.

Changing a small part of the site and testing will give you confidence to roll out changes across an entire site.

#2 Avoid Costly Mistakes

When running an SEO test you can quickly see if a hypothesis is incorrect and the results have gone south!

If it is conclusive over a few days that a change to page title and meta description has led to a decrease in click through rate, reverse the change and continue to track the CTR in the test to see if it recovers.

#3 Discover Opportunities

Brad Smith, the founder of Codeless and an author at WordStream, puts it well in his own piece about SEO testing ideas:

If you’re a smaller player in the search market, Rand agrees that the strategy of doing what bigger brands don’t do is something you should take to heart.

Running tests and experiments is a prime way to take on larger incumbent websites who aren’t as nimble and can’t make changes quickly.

If you are one of these larger established websites – building out a platform so your website can quickly rollout page and content changes to adopt a test-driven approach is critical to not having your lunch stolen from bottom up competitors.

#4 Base Decisions On Data Instead Of Opinions

Using a framework for running SEO tests will allow you to base decisions on data instead of opinions.

We’ve already said that the world of SEO consultants offer up a myriad of different opinions – some based on actual experience but often quoting someone else (who in turn has quoted someone else!). This can often lead to the game of SEO Chinese Whispers!

Running experiments and tracking test results allows you to report on what actually matters to your website and make decisions based on data rather than opinions.

#5 Justify Retainers And Ongoing Projects

If you are an agency you can be on the constant merry go round of proving the value you offer to clients to get retainer contracts renewed.

Using a test-driven SEO approach you can use the hypothesis and testing framework as part of your monthly deliverables. Committing to run a number of experiments each month and reporting on the results can show how you are delivering value to clients, growing organic traffic, and also avoiding costly mistakes.

For internal SEO teams, SEO testing helps you show the efforts deployed each month to your website, what has worked and can be rolled out, or what hasn’t and can be scaled back. This can help the business invest into the areas of the website they are confident will provide a return on that investment.

6 Best Practices For SEO Testing

You are going to be making big decisions based on the results you gather from SEO tests so it’s important to run your experiments in the right way. Checkout these best practices we have put together from running and analyzing thousands of different tests across many websites and niches.

#1: Pick Pages And Keywords That Already Get Impressions

When you are looking to optimize around a keyword or page, ensure there is enough search traffic to allow the test to show results.

If a keyword only gets an estimated 10 searches per month, you aren’t going to get a statistically significant result even if you do dramatically improve the ranking.

Although an improvement in ranking is nice, if it drives a limited amount of new traffic to your site, it is not going to move the needle.

#2: If You Can, Run Split Tests

AB split testing for SEO is slightly different to AB split testing for Conversion Rate Optimization (CRO).

With CRO you take the same page, but show different versions of it to different people. You then measure which version of the page gets the best result you are optimizing for.

It’s impossible to do this with an SEO split test as you can’t (or at least don’t want to) have two very similar versions of the same page indexed by Google.

The way around this is to create 2 groups of similar pages. These could be blog posts, product pages or category pages. One group of pages would be your test group and in this group you would make the change you wish to test. The second group would be the control group. You do not make any changes to this group of pages.

You can then compare the two groups, for a period before the change was made to the test group, to how the test group performs compared to the control group after the change was made.

Algorithm updates and seasonality factors always need to be taken into account when running a test that compares two different time periods.

Running a split test will help remove these factors from your results as both groups of pages would be affected by them. Therefore any increase or decrease in key metrics can be attributed directly to the changes being tested.

Split tests take more effort to set up as you are changing a group of pages rather than a single individual page, but the results will be more reliable as you will be removing Google algorithm updates and seasonality from the test results.

#3: Form A Clear Hypothesis

If you go into Google Analytics without a clear question you’re trying to answer, you’ll likely get distracted before you find something useful.

Similarly, an SEO test will only be successful if you started with a clear hypothesis you’re trying to prove or disprove. One example would be:

Test hypothesis example: Adding action verbs into the meta titles of service pages increases organic CTR by more than 2 percentage points, which will lead to higher average positions and a proportionate growth in organic clicks.

This hypothesis would provide for an actionable SEO test because it is specific in both what we want to do, what effect we think it’ll achieve, and how we measure the effect.

#4: Test One Thing At A Time

One of the core principles of testing anything is to only test one thing at a time. If you intentionally make more than one change, the test becomes invalid because you don’t know which of your steps was the one that caused the change in performance.

Ceteris paribus – other things equal.

You know it’s important when there’s a Latin phrase for it.

#5: Know What You Are Trying To Optimize For

The query you want to improve around and the number of searches this query gets should dictate which metric to try and improve.

If you are targeting a low search, but highly valuable keyword to your business, you want to directly improve the position of the page targeting this keyword in Google. This would suggest content and onpage changes.

If you are targeting a keyword that gets a high volume of searches, and the target page already ranks reasonably well resulting in a decent number of daily impressions in the search result page, working on page titles, meta descriptions or structured data that can be pulled through and displayed in the results pages to improve CTR would be our suggestion.

#6: Set The Right Test Duration

What you are testing should dictate how long you want to run a test for:

• Page title and meta description changes : 2 weeks
• Content improvements : 4 weeks
• New links : 6 weeks

If you are not working on a deadline for particular results, I would always run tests for 6 weeks each time.

There is no harm in capturing data over a longer period of time.

But if you are working on a retainer basis and want to move onto the next set of tests each month, you may want to stick to the shorter time frames above for the suggested changes.

Ideas For Test-Driven SEO Experiments You Can Try

Get inspired by our list below containing a big variety of experiments you can play with. Please remember that each of them can lead to both positive and negative changes in your organic clicks – many times you’d be surprised. This is why it’s important to test everything.

The list is a result of our own experience combined with ideas published by others. We’d like to give thanks and shout-outs to seoClarity, WebFX, Moz, and WordStream.

A: SERP Appearance

1, Meta Titles And Descriptions

• We’ve written an entire article on how to SEO test Page Titles we recommend reading.
• Which keywords and phrases should we rewrite them for?
• Which keyword variations have higher CTR?
• Entice curiosity vs. give more away before users even click?
• Optimize for CTR primarily vs. optimize for position primarily?
• Do action verbs in meta translate to higher conversion intent and conversion rates on-site?

2, Date Stamps & Publish Dates

• Show them or not?
• If we do show them, does it pay off to make small changes that enable updating the publish date?

3, Structured Data

• Do they help us with position and CTR?
• Do they help us show in position #0?
• Focus on industry-specific snippets – recipe ingredients or concert dates, vs. more generic snippets – reviews or page type?

B: On-Page Copy

4, Copy Length

• Redoing the article into a skyscraper resource vs. breaking it up into a few laser-focused 1,200-word articles that solve a quick need?
• Do skyscrapers need to be at least as long as the competing pages in positions 1-3?

5, Titles and Headings

• Does stating the featured questions in headings lead to higher rankings even if we don’t get featured in question boxes?
• Include semantic variations of the focus keyword in each heading?
• Make them more narrative vs. emotional vs. action-focused? Longer vs. shorter?
• Optimize for a second focus keyword?
• How impactful is the list format?

6, First 100 Words

• Add internal links?
• Add external links?
• Make them more similar to the meta description?
• Add that second focus keyword?
• State the featured questions vs. answers to the featured questions?

7, Sections

• Which ones are not relevant anymore and due for removal?
• Add new ones based on semantic keywords vs. start a new article for the purpose?
• Add opinions that challenge the one presented by the author?
• Add a case study vs. create a standalone case study post?

8, Readability And Skimmability

• Break up sentences for higher readability scores?
• Use humor and plant copywriting gems to encourage reading instead of skimming?
• Add table of contents with anchor links to H2s?
• Add H3s to the table of contents and make the table of contents longer?

C: Visuals

9, Images

• How many contiguous pixels are OK without an image or visual?
• What type of images are best – data visualizations vs. gifs vs. screenshots?
• Does it help engagement and rankings to include visuals that stop the user in their scrolling? Carousels vs. click-activated sliders?
• Do infographics that replace text help or hurt rankings?
• Infographics vs. videos?
• How impactful are image descriptions (Pinterest case study of SEO testing)?

10, Videos

• How effective are videos at helping us show in featured snippets for how-to queries?
• Place them at the top vs. at the bottom of the page?
• Recap the article vs. focus on one aspect such as a use case?
• Host on YouTube vs. Vimeo vs. Wistia vs. custom?

11, Expert Quotes

• Break up text with image vs. expert quote?
• Reiterate your own line vs. chime in with a new one?

12, Custom

• Can we make interactive explainers, such as this one on stand-up comedy or this one on group dynamics in avalanche terrain, and still make sure that the page is easy to understand by search engines?
• Do dynamic visualizations, such as this social distancing simulator, grow organic clicks more than if the visualizations were static?

D: UX & Design

13, Page Speed

• Focus on mobile vs. the whole site?
• Which elements do we limit on mobile?
• Is time to first byte more impactful than page load speed?

14, Design

• Optimize color scheme for readability?
• Move elements around for a new layout?
• Show estimated read time?
• Implement additional scroll controls?

15, Interaction

• Optimize keywords in calls to action (CTA)?
• Streamline our in-line CTAs?
• How many related articles do we feature?
• Turn blog comments on or off?
• Display social share stats?
• Display social share stats once they reach a certain amount?

E: Information Architecture

16, Internal Links

• Optimize for cornerstone structure in our main topic?
• Feature blog articles on home page?
• Optimize anchor text for new keyword data?
• Place cornerstone links in the first 100 words vs. article body?
• Add vs. remove footer links?
• Expand vs. streamline main nav?
• Which pages need fewer clicks from home?

17, Content

• Update and upgrade existing articles vs. add new pages?
• Turn an article into a service page?
• Remove or rewrite thin content?

F: Indexability & Crawling

18, Trailing Slashes

• What are the effects of unifying all URI’s to having a slash at the end?

19, Noindexing

• Noindex vs. add a canonical URL?
• Noindex all pages that don’t rank vs. only the pages we don’t care about anymore?

20, Sitemaps

• Remove sitemaps that contain low quality pages?
• Submit sitemaps manually vs. let Google fetch them?
• Change sitemap plugin?
• Audit redirects and update to valid links?

G: Distribution & Backlinks

21, Bringing Traction To The New Post

• Send it to peers you worked with vs. your mastermind Slack community vs. your newsletter list vs. somebody else’s newsletter list vs. Reddit vs. all of it?
• Post to Twitter vs. LinkedIn?
• 1,000 clicks from organic social vs. 1,000 clicks from paid social?

22, Backlinks

• Link to experts first then ask to return the favor vs. make agreements first?
• Link exchange vs. unidirectional link placement?
• Place on a large news site vs. a small site relevant to the topic?
• Keep anchor text natural vs. optimize it for the keywords?
• Place one backlink in the first 100 words vs. two backlinks lower down in the guest post body?

The intention of this list is to be extensive and inspire you with at least a few tests you hadn’t thought of before. But we do not operate under the illusion that it is complete. You, reading this right now, have probably thought of a few more experiments with potential for impact. Please let us know and we’ll carefully consider including it!

How To Track Results

There are two options when it comes to tracking results of your SEO experiments:

1, Manually track the results in a spreadsheet such as Google Sheets

2, Use an automated tool such as SEOTesting.com to track and report on the results for you.

If you are running a single time based test, it is totally possible to report on this using a spreadsheet. You’ll only be looking up a single url or query in Google Search Console each day so you can record the impressions, clicks, average position and CTR.

We’ve setup a sample Google Sheet that demonstrates an SEO test we ran and the results graphed. You can make a copy of this spreadsheet to base your own results around.

If you are running multiple time based tests, or running a split test, which by its nature is comprised of multiple urls, using a tool such as SEOTesting will save you many hours each week across all your sites, experiements and urls.

We’ll now guide you through how to setup a time based SEO test, and SEO split test.

How To Setup An SEO Test In SEOTesting.com

1, Make the change to the page in question that you wish to test.

2, Within Google Search Console, request that the page is reindexed.

3, Log in to SEOTesting.com and select the site you wish to work with from the dashboard.

4, Click on Simple Tests on the sidebar, and then “Create a new test” button.

5, You’ll now see the main screen to create a test. Make sure to give it:

• A meaningful title
• A full description of the hypothesis you are testing. This will be important when reviewing results in the future.
• Select the type of thing you want to test : Page, Query or Site
• Enter the url, or query to test if applicable.

6, Select the date the change was made. By default it will pre-select to today, but you can move this back if you made the change a while ago. You can even create tests where the control and test periods are in the past – so you can retrospectively run SEO tests against changes you made.

7, Set the test period – this will be how many weeks before the change, and after, you want to compare. As a reminder these are the minimum we suggest:

• Page title and meta desc changes – 2 weeks
• Content improvements/changes – 4 weeks
• Test new links acquired – 6 weeks

8, Upon clicking “Create” you’ll be asked to confirm the dates for the control and test periods. These are based on the “change date” and “test period” selected. You probably won’t ever have to fiddle with these and can simply click on the “confirm dates” button.

Once confirming the test dates, your test is now created and awaiting pre-processing.

If both your control and test periods are in the past – you should get an email stating your results are ready to view within 15-30 minutes.

If your change was made today, your test will pre-process, but it will take a couple of days for the test period data to start rolling in. This is because the data provided by Google Search Console is always at least 2 days behind.

Viewing And Interpreting Results

This is what you are hoping to get from your test results:

All indicators for the test period are green, and the graphs for the test period are up.

As long as there has not been a massive algorithm shift or seasonal event in the test period you can be confident your changes have improved the rank and clicks for this test.

If there is a mixture of green and red indicators your test results will need a bit more consideration.

If impressions and clicks is green, but CTR is red, you can consider this test a success. If the page starts ranking for more queries the number of impressions it gets it likely to go up, and while the clicks may also go up, they may not correlate directly with impression growth. Therefore the CTR might go down.

How To Setup An SEO Split Test

You need two groups of pages for SEO split tests:

• Contol Group : A group of pages that you keep the same
• Test Group : The group of pages where you make the change you are testing.

1, Make the change to your test group pages, and submit those urls to Google for reindexing.

2, Log into SEOTesting.com and click on Split Tests from the side menu. Then click on “Create new Split Test”

3, You’ll now see the screen to create a new Split Test. You’ll need to give it:

• A meaningful title
• A full description of the hypothesis you are testing. This will be important when reviewing results in the future.

4, Set the date the changes to the test pages were made. By default it is set to today, but you’ll notice you can set the date to be in the past, so you can retrospectively run split tests.

5, Next you need to copy and paste the control and test urls into the relevant textboxes. Each url should go on a separate line.

6, Select the test period you want to run this test for.

7, You can also tick the checkbox for an “On Going Test”. This means the split test will have no end date, which is really useful if you want to continually track how two groups of pages are performing against each other.

8, Finally you can enter a 3 character ISO country code if you want to filter the results.

Once you click the “Create” button your split test will go into the queue to be pre-processed. This pre-processing will gather any historical data it can, and if the start date of the test was in the past, any result data will also be collected.

Viewing And Interpreting Results

With SEO split tests we want to look at the click difference between the test group and the control group.

The pre-processing of the test calculates [avg test group clicks] – [avg control group clicks] each day.

We then use those daily differences to create a baseline during the period before the test starts. We do this as it is highly unlikely the [test group clicks] – [control group clicks] equals 0 each day.

Our baseline is the blue line before the test period begins.

Once the test begins we continue to calculate the daily differences between the test group and control group, and from those values we calculate a new average difference – this is the black line.

A positive result for your split test would be your test group getting an increase in clicks when compared to the control group. This would be indicated by the black average line being higher than the blue line.

If the black line appears below the pre-test blue average, you can consider the test having a negative result – as in the test group is getting less clicks compared to the control group. You may well want to roll back the changes you made to the test group.

As well as looking at the click difference graph SEOTesting.com also plots the daily sum of the clicks, impressions as well as the average position and click through rates of the test group and control group.

These graphs offer further insights than just an improvement in clicks. If your hypothesis and test changes are around content improvements, but your clicks remain the same and yet you see your test group gaining an increase in position and impressions, it would be good to do some further testing around page title and meta description optimization.

Summary

Hopefully this guide has given you a good overview of why SEO testing should be an important part of growing your organic search traffic from Google.

With the business cases, best practices, ideas to test and how to track results, hopefully we have given you everything you need to get going.

If you’d like to give SEOTesting.com a try – we have a 14 day trial, and we’d be delighted to show you what the tool can do to help you.