A/B Testing Best Practices To Maximize Results

Written by Marc Alringer

The best way to figure out what drives the most engagement and conversions on your website is to conduct A/B testing. These tests will help you determine exactly how customers respond to specific elements on your website. A/B testing, which is also known as split testing or bucket testing, is an experiment to decide which version of a webpage, app, or campaign are most effective. 

To conduct A/B tests, traffic to a page is split into groups where visitors see slightly different versions of the same page. A/B testing helps make data-driven decisions on whether a particular campaign, or it’s features, is outperforming another.

For example, if you want to determine which button placement will receive a better clickthrough rate, you can run an A/B test. Version A would have the button at the top of the web page and Version B would have that same button on the sidebar. Based on your traffic, you may want to run the test for a few hours or a couple days before jumping to conclusions. Use the data to determine which button placement received the larger number of clicks. 

Whether you want to try something new to increase your revenue or you’d like to see if your current website is optimized, you should implement A/B testing. To carry out these tests effectively, here are some A/B testing best practices to maximize results. 

General A/B Testing Best Practices: 

Sample Size

One of the A/B testing best practices to maximize results is to make sure that you perform your experiment on a large enough sample size to obtain valuable results. If you don’t perform A/B tests on enough people, then your results might not be statistically significant and any data-driven decisions will be inaccurate. 

To find out the ideal sample size, you can use Optimizely’s sample size calculator. This tool will ask for your current conversion rate and then you’ll enter the percentage increase you’d like to achieve. With these numbers, the tool will derive the appropriate sample size required to conduct successful A/B tests. 

A/B testing
Image Author: Seobility License: CC BY-SA 4.0

Control Variable 

Always make sure that you have a control group. This allows you to compare results to a baseline so you can determine if your modification had a positive impact. 

It’s also important to test a few versions at once. A/B testing best practices recommends that you test at least 2 variations at a given time. However, only test variations on one variable at a time. 

If you test more than one variable, you won’t be able to determine which variables influenced the outcome. For example, if you were to test the color of a button and the text in a headline, and experienced a significant spike in conversions, you wouldn’t know which variable to attribute the results to.    

Although you may be only testing one variable, there are a number of different ways to size, place, and color that one variable. For example, when testing a button on a web page, there could be 5 color variations. Then you can determine which color had the greatest impact on conversions or click-throughs. 

Significant Results 

Before implementing the results from your A/B tests into any of your strategies, you’ll want to make sure that your data is reliable. This means you’ll want the data to have statistical significance to make sure your results aren’t due to random chance. 

a/b testing data

Test Simultaneously and Determine Your Test Duration

You should run your tests simultaneously but make sure you’re only testing for one element at a time. For example, if you want to test the headline and the call-to-action, you should conduct these as two separate tests but test the variations at the same time. 

The test durations contribute to the readability of your results. Depending on how many conversions you want and how many variants you’re testing, the duration of your test should vary. Digital Marketer provides the following chart to determine the appropriate length for your  testing period. 

working women

For example, if you are testing 4 variants on the color of your call-to-action button and you’re getting 58 conversions per day, then your test period should be 7 days. You should increase your testing period based on the number of conversions you currently experience per day. The lower the number of conversion, the larger the testing period because you’ll need to allow your test to collect more data.

SEO A/B Testing 

SEO helps your business increase its visibility on search engines, ultimately driving more traffic to your website. Although Google encourages A/B testing, it’s important to make sure you’re not jeopardizing your SEO efforts. To ensure that you don’t negatively impact your search rankings there are a few things you can do:  

  • Open your test to Google’s search engine robots
  • If there’s multiple URLs, use rel=canonical tag 
  • Site speed: make sure the sites load synchronously 
  • Put a 301 redirect from the original page to the variations 

Common A/B Testing Elements

You might wonder what you should test in the first place. Before selecting a test element, you should understand the purpose behind testing it. What are you trying to improve and why? The element chosen should have an end goal of contributing to lead generation. You should begin with the factors that will drive the most conversions for your business once optimized appropriately.

Here are some common A/B testing elements: 

  • Contact form button color/font/copy

  • Heading copy

  • Email subject line 

  • Visual media 

  • Content length 

  • Social media icon placement

  • Sales copy or product descriptions 

A/B testing is a direct way to obtain data from your website visitors. It helps you determine what’s working and what’s not. Follow these A/B testing best practices to maximize results and boost your website performance. If your company needs any assistance with improving the overall performance of your website, contact our team today!

Marc Alringer
Written by
Marc Alringer
President/Founder, Seamgen
I founded Seamgen, an award winning, San Diego web and mobile app design and development agency.




Text us
We’re ready for you! Fill out the fields below and our team will get back to you as soon as possible.