Written by Nick Swan. Published on 02, June 2020
How do you prove the work you are doing to a website is offering a return on investment by getting better rankings and more clicks from Google?
One way is implementing a test-driven SEO process of:
Change -> Test -> Learn
This allows you to pinpoint what is working, what isn’t working, and how the business should invest in it’s SEO strategy going forward.
In this guide we will define what SEO testing is and describe valuable use cases. We’ll cover best practices, how to track results, and cover an extensive list of ideas for SEO experiments you could run.
Finally we’ll walk you through how to setup an SEO test using SEOTesting.com.
Table of contents
SEO testing is the process of measuring and evaluating the results of a change you make to a page or your site in relation to organic search traffic.
We look at clicks, impressions, position, click through rate, and the number of queries ranked for. This can be based on an individual page or section of a site over two time periods, a specific query, two separate groups of pages you run as a split test, or a group of pages you continually track to help you show your work is bearing fruit.
This brings us nicely to why we should be running SEO Tests…
SEO testing shows you what actually works.
Google’s ranking algorithm is a black box. In fact there are multiple algorithms that are fed into other algorithms, that come together to produce an overall ranking of a page or website.
There are over 200 ranking factors the algorithms use, with machine learning working alongside these signals to try and show the best results it can for the user's individual query. It has been reported that the Google search team are unsure of the weight of individual signals as machine learning has taken over judging which are the most important.
Different niches are also evaluated against different criteria and factors. Travel websites are likely to use different ranking factors to medical websites - and potentially completely different ranking algorithms.
Spend an hour on Twitter and you’ll get a hundred different opinions on what it takes to improve rankings in Google. All these opinions may have worked in the past - but are they still applicable, and relevant to a site in your niche?
Running a website change as an SEO test allows you to evaluate these ideas and see whether they work on your site and in the niche it sits.
Once you see a positive result you can confidently roll out the change to other pages. If the results aren’t positive you can roll the change back.
You can then move onto the next thing to optimize and test.
Systematic testing turns ad-hoc website changes into a methodical approach of collecting insight about what works for a particular website. When done long term, test-driven SEO tends to lead to a snowball effect, often visualized by exponential growth curves.
Sometimes there is a bit of confusion over what is an SEO audit and what is an SEO test.
An SEO audit is the process of looking at the current state of an existing site or page, and based on technical best practices and rule sets, coming up with a list of things to fix, improve or change.
SEO site audits are valuable because they review the overall state of your website. They may discover inconsistencies in your URI structure, pages that should be noindexed or have canonical URLs, images that are missing alt tags or have disproportionate file sizes. SEO audits can even discover content gap opportunities or information architecture flaws.
An SEO test is the process of tracking the results, since a change has been made. Quite often, you should look to do an audit, and run SEO tests on the fixes the audit suggests. This way you can evaluate the effectiveness of the recommended changes.
SEO testing is the super power that can help internal SEOs and agencies demonstrate the value and return of their work. Building a test-driven SEO approach into your work practices allows you to have a continuous method of improvement to make sure your website does not stagnate.
We've outlined 5 business cases to help you propose a test-driven SEO approach to project owners.
It can be costly to implement a change across an entire site, and it can also be risky if you do not know how the change will affect rankings.
By using an SEO testing approach you can make your change to a single page, or a group of pages, and track the results.
Upon getting a positive result, you can roll the change out across the site, or to a larger test group if you want to make sure the initial test results replicate.
A good SEO test gets rid of all "guesswork" when it comes to making significant changes to content. We all know making big changes to your content can have severe consequences, and by approaching this as an iterative process, you can test something small and get feedback on the outcome and adjust accordingly. Then you can scale it with minimal effect on your SEO while cutting cost and unnecessary stress.
Changing a small part of the site and testing will give you confidence to roll out changes across an entire site.
When running an SEO test you can quickly see if a hypothesis is incorrect and the results have gone south!
If it is conclusive over a few days that a change to page title and meta description has led to a decrease in click through rate, reverse the change and continue to track the CTR in the test to see if it recovers.
Brad Smith, the founder of Codeless and an author at WordStream, puts it well in his own piece about SEO testing ideas:
If you want to grow site traffic in a serious way, you need to experiment, do what big brands won’t do, and discover what works best for you (instead of reading someone else’s best guess.)
If you're a smaller player in the search market, Rand agrees that the strategy of doing what bigger brands don't do is something you should take to heart.
Running tests and experiments is a prime way to take on larger incumbent websites who aren't as nimble and can't make changes quickly.
If you are one of these larger established websites - building out a platform so your website can quickly rollout page and content changes to adopt a test-driven approach is critical to not having your lunch stolen from bottom up competitors.
Using a framework for running SEO tests will allow you to base decisions on data instead of opinions.
We've already said that the world of SEO consultants offer up a myriad of different opinions - some based on actual experience but often quoting someone else (who in turn has quoted someone else!). This can often lead to the game of SEO Chinese Whispers!
Running experiments and tracking test results allows you to report on what actually matters to your website and make decisions based on data rather than opinions.
If you are an agency you can be on the constant merry go round of proving the value you offer to clients to get retainer contracts renewed.
Using a test-driven SEO approach you can use the hypothesis and testing framework as part of your monthly deliverables. Committing to run a number of experiments each month and reporting on the results can show how you are delivering value to clients, growing organic traffic, and also avoiding costly mistakes.
Running SEO tests allows you to demonstrate incremental organic traffic. These traffic gains can lead directly to additional revenue or micro-conversions that can lead to future revenue. It might only take 1-2 successful tests to generate enough incremental revenue to justify several months of an SEO retainer costs.
For internal SEO teams, SEO testing helps you show the efforts deployed each month to your website, what has worked and can be rolled out, or what hasn't and can be scaled back. This can help the business invest into the areas of the website they are confident will provide a return on that investment.
You are going to be making big decisions based on the results you gather from SEO tests so it's important to run your experiments in the right way. Checkout these best practices we have put together from running and analyzing thousands of different tests across many websites and niches.
When you are looking to optimize around a keyword or page, ensure there is enough search traffic to allow the test to show results.
If a keyword only gets an estimated 10 searches per month, you aren't going to get a statistically significant result even if you do dramatically improve the ranking.
Although an improvement in ranking is nice, if it drives a limited amount of new traffic to your site, it is not going to move the needle.
AB split testing for SEO is slightly different to AB split testing for Conversion Rate Optimization (CRO).
With CRO you take the same page, but show different versions of it to different people. You then measure which version of the page gets the best result you are optimizing for.
It's impossible to do this with an SEO split test as you can't (or at least don't want to) have two very similar versions of the same page indexed by Google.
The way around this is to create 2 groups of similar pages. These could be blog posts, product pages or category pages. One group of pages would be your test group and in this group you would make the change you wish to test. The second group would be the control group. You do not make any changes to this group of pages.
You can then compare the two groups, for a period before the change was made to the test group, to how the test group performs compared to the control group after the change was made.
Algorithm updates and seasonality factors always need to be taken into account when running a test that compares two different time periods.
Running a split test will help remove these factors from your results as both groups of pages would be affected by them. Therefore any increase or decrease in key metrics can be attributed directly to the changes being tested.
Split tests take more effort to set up as you are changing a group of pages rather than a single individual page, but the results will be more reliable as you will be removing Google algorithm updates and seasonality from the test results.
If you go into Google Analytics without a clear question you're trying to answer, you'll likely get distracted before you find something useful.
Similarly, an SEO test will only be successful if you started with a clear hypothesis you're trying to prove or disprove. One example would be:
Test hypothesis example: Adding action verbs into the meta titles of service pages increases organic CTR by more than 2 percentage points, which will lead to higher average positions and a proportionate growth in organic clicks.
This hypothesis would provide for an actionable SEO test because it is specific in both what we want to do, what effect we think it'll achieve, and how we measure the effect.
One of the core principles of testing anything is to only test one thing at a time. If you intentionally make more than one change, the test becomes invalid because you don’t know which of your steps was the one that caused the change in performance.
Ceteris paribus - other things equal.
You know it's important when there's a Latin phrase for it.
The query you want to improve around and the number of searches this query gets should dictate which metric to try and improve.
If you are targeting a low search, but highly valuable keyword to your business, you want to directly improve the position of the page targeting this keyword in Google. This would suggest content and onpage changes.
If you are targeting a keyword that gets a high volume of searches, and the target page already ranks reasonably well resulting in a decent number of daily impressions in the search result page, working on page titles, meta descriptions or structured data that can be pulled through and displayed in the results pages to improve CTR would be our suggestion.
What you are testing should dictate how long you want to run a test for:
If you are not working on a deadline for particular results, I would always run tests for 6 weeks each time.
There is no harm in capturing data over a longer period of time.
But if you are working on a retainer basis and want to move onto the next set of tests each month, you may want to stick to the shorter time frames above for the suggested changes.
Get inspired by our list below containing a big variety of experiments you can play with. Please remember that each of them can lead to both positive and negative changes in your organic clicks - many times you'd be surprised. This is why it's important to test everything.
The intention of this list is to be extensive and inspire you with at least a few tests you hadn’t thought of before. But we do not operate under the illusion that it is complete. You, reading this right now, have probably thought of a few more experiments with potential for impact. Please let us know [in the comments below / via this email address / twitter post permalink / twitter handle] and we’ll carefully consider including it!
There are two options when it comes to tracking results of your SEO experiments:
1, Manually track the results in a spreadsheet such as Google Sheets
2, Use an automated tool such as SEOTesting.com to track and report on the results for you.
If you are running a single time based test, it is totally possible to report on this using a spreadsheet. You'll only be looking up a single url or query in Google Search Console each day so you can record the impressions, clicks, average position and CTR.
We've setup a sample Google Sheet that demonstrates an SEO test we ran and the results graphed. You can make a copy of this spreadsheet to base your own results around.
If you are running multiple time based tests, or running a split test, which by its nature is comprised of multiple urls, using a tool such as SEOTesting will save you many hours each week across all your sites, experiements and urls.
We'll now guide you through how to setup a time based SEO test, and SEO split test.
1, Make the change to the page in question that you wish to test.
2, Within Google Search Console, request that the page is reindexed.
3, Log in to SEOTesting.com and select the site you wish to work with from the dashboard.
4, Click on Simple Tests on the sidebar, and then "Create a new test" button.
5, You'll now see the main screen to create a test. Make sure to give it:
6, Select the date the change was made. By default it will pre-select to today, but you can move this back if you made the change a while ago. You can even create tests where the control and test periods are in the past - so you can retrospectively run SEO tests against changes you made.
7, Set the test period - this will be how many weeks before the change, and after, you want to compare. As a reminder these are the minimum we suggest:
8, Upon clicking "Create" you'll be asked to confirm the dates for the control and test periods. These are based on the "change date" and "test period" selected. You probably won't ever have to fiddle with these and can simply click on the "confirm dates" button.
Once confirming the test dates, your test is now created and awaiting pre-processing.
If both your control and test periods are in the past - you should get an email stating your results are ready to view within 15-30 minutes.
If your change was made today, your test will pre-process, but it will take a couple of days for the test period data to start rolling in. This is because the data provided by Google Search Console is always at least 2 days behind.
This is what you are hoping to get from your test results:
All indicators for the test period are green, and the graphs for the test period are up.
As long as there has not been a massive algorithm shift or seasonal event in the test period you can be confident your changes have improved the rank and clicks for this test.
If there is a mixture of green and red indicators your test results will need a bit more consideration.
If impressions and clicks is green, but CTR is red, you can consider this test a success. If the page starts ranking for more queries the number of impressions it gets it likely to go up, and while the clicks may also go up, they may not correlate directly with impression growth. Therefore the CTR might go down.
You need two groups of pages for SEO split tests:
1, Make the change to your test group pages, and submit those urls to Google for reindexing.
2, Log into SEOTesting.com and click on Split Tests from the side menu. Then click on "Create new Split Test"
3, You'll now see the screen to create a new Split Test. You'll need to give it:
4, Set the date the changes to the test pages were made. By default it is set to today, but you'll notice you can set the date to be in the past, so you can retrospectively run split tests.
5, Next you need to copy and paste the control and test urls into the relevant textboxes. Each url should go on a separate line.
6, Select the test period you want to run this test for.
7, You can also tick the checkbox for an "On Going Test". This means the split test will have no end date, which is really useful if you want to continually track how two groups of pages are performing against each other.
8, Finally you can enter a 3 character ISO country code if you want to filter the results.
Once you click the "Create" button your split test will go into the queue to be pre-processed. This pre-processing will gather any historical data it can, and if the start date of the test was in the past, any result data will also be collected.
With SEO split tests we want to look at the click difference between the test group and the control group.
The pre-processing of the test calculates [avg test group clicks] - [avg control group clicks] each day.
We then use those daily differences to create a baseline during the period before the test starts. We do this as it is highly unlikely the [test group clicks] - [control group clicks] equals 0 each day.
Our baseline is the blue line before the test period begins.
Once the test begins we continue to calculate the daily differences between the test group and control group, and from those values we calculate a new average difference - this is the black line.
A positive result for your split test would be your test group getting an increase in clicks when compared to the control group. This would be indicated by the black average line being higher than the blue line.
If the black line appears below the pre-test blue average, you can consider the test having a negative result - as in the test group is getting less clicks compared to the control group. You may well want to roll back the changes you made to the test group.
As well as looking at the click difference graph SEOTesting.com also plots the daily sum of the clicks, impressions as well as the average position and click through rates of the test group and control group.
These graphs offer further insights than just an improvement in clicks. If your hypothesis and test changes are around content improvements, but your clicks remain the same and yet you see your test group gaining an increase in position and impressions, it would be good to do some further testing around page title and meta description optimization.
Hopefully this guide has given you a good overview of why SEO testing should be an important part of growing your organic search traffic from Google.
With the business cases, best practices, ideas to test and how to track results, hopefully we have given you everything you need to get going.
If you'd like to give SEOTesting.com a try - we have a 14 day trial, and we'd be delighted to show you what the tool can do to help you.