A/B testing is a great way to improve your website. It's an easy way to make sure that the changes you make have a real impact, and it can help you objectively decide which changes are worth implementing. In this article, we will walk you through 5 simple steps to create and run an A/B test on your website that helps you achieve your performance goals.

What is an A/B test?

Simply put, A/B testing is a method of comparing two versions of a webpage to see which one performs better. An A/B test is used to improve the conversion rate of your website or landing page, and it can be used for almost any kind of conversion metric: Clicks, sales, leads generated or sign-ups.

There are plenty of tools out there that will help you run an A/B test on your website or landing page. For example: Optimizely (a cloud-based tool), VWO (a cloud-based tool) and Google Optimize (natively integrated with Google Analytics). Take a look at the visual example below to get an idea of how you can change different elements of your website in an A/B test.

Why AB testing?

A/B testing is a simple and effective way to test new ideas and understand what works and what doesn’t. It helps you learn more about your customers, while also providing insight into what they want from your website.

Let's look at an example: You have two versions of an email signup form on your website: one with just text and one with both text and graphics. Which one should you use?

Of course, it’s very difficult to know which method will work best for getting people on board. Often when deciding on an approach, a lot of opinions get thrown about and it’s hard to think objectively about what will best help you reach your end-goal. So, instead of debating endlessly and going back-and forth, you can set up an A/B test.

1. Pick what you want to test

When it comes to A/B testing, the first step is understanding what you want to test. Think about what kind of knowledge you could gain that could add significant value. One of the most important elements of a good A/B test is a clear, concise hypothesis that can be objectively tested and will link directly to your overarching business goals.

This is a good time to be creative and think of something that will ‘move the needle’. It’s also important that you can measure it—so if you want to test whether buttons are more effective when they are green or blue, for example, make sure you have some way of measuring how many people clicked on each button.

When you're trying to find the right A/B test to run, you need to first consider what your website is trying to accomplish. Is it a SaaS tool? An e-commerce store? A blog that wants traffic and social media shares? You can use an A/B test as an opportunity to improve several different things:

• Navigation: Do you find that people have trouble finding information on your website? Could they be saving time by accessing certain resources through different menus? Try creating new navigation menus and see how they perform compared with the old ones.
• Content: Do users understand what your site is about or do they get confused by terms or language used? If so, consider using different text and visuals on your website to see what performs better on your key metrics.
• CTA (Call To Action): Does your page layout encourage visitors into taking action and converting into customers? Try changing up where call-to-action buttons are placed and how you guide users to them.

If the goal of your website is lead generation, for example, then an A/B test might involve changing the copy on one page from "Submit" to "Contact us" with information about how many people have already contacted them so far this month in order to encourage visitors who haven't yet done so yet (but may be interested). This is an effect known as social proof. Or perhaps if you’re an ecommerce site, you might want to experiment with using the scarcity effect. The scarcity effect works by creating the feeling that people need to take action now, otherwise they will miss out. This could be implemented with something as simple as an indicator that shows which products are almost out of stock.

2. Determine your testing methodology

Now you’ve determined what you want to test and why it’s important to your business. It’s time to begin the next step - how are you going to implement the test? Let’s say for example that you want to change some of the product visuals on your e-commerce website. You predict that by making the image of the product larger that it will receive more attention and therefore increase product sales. Your next step is to define which images you want to change, and of which product(s). If you change all the product images on your site, then you’re of course not going to be able to tell if making the visuals bigger actually made a difference to your sales. So you could decide to keep it simple and make the images bigger of a few key products that are listed on your site on separate pages.

In A/B testing, it’s crucial to ensure that you only change one variable at a time. If you change multiple elements of your website and/or page, you won’t be able to distinguish between which change actually affects your conversion metrics. Therefore making it harder to derive any meaningful conclusions that you can use in the future.

How long should you run an experiment for?
When deciding how long to run an experiment for, you'll need to consider a few things. The longer you run an experiment for, the more reliable your results will be. A good rule of thumb is that if your site has more than 100 visitors per day, then it should be at least 14 days long. This allows enough time for users to see both variations of your site and gives enough time for data to accumulate in order to get accurate results.

Finding the exact right length can be tricky, as longer tests also create an opportunity cost. If you’re testing one small feature for 2 months, that also means that you cannot test other features on that page for 2 months until the first experiment is over. Otherwise you might affect the validity of your experiment results. However, running too many tests can also be expensive - you need to research and find ideas for things to test, design the changes and implement them on the live site. This can mean several hours from developers and designers in the process. And of course, not every experiment results is a positive one, so you need to pick wisely!

So what’s the best length for your A/B test? It depends. If you're looking for a quick answer, then you should be aiming for at least 14 days. This will give you enough time to gather data and get accurate results while also minimizing costs. However, this does depend greatly on the regular traffic to your website.

3. Choose your A/B testing tool

Having a good hypothesis and methodology is great, but they’re not much use if you don’t have the tools to implement them! Fortunately, there are many solid A/B testing tools on the market. In order to choose the right one for your needs, you must first look at the following factors:

Firstly (and most importantly) does the functionality meet the needs of your experiment? Let’s say that our hypothetical A/B tester wants to change the navigation of their website, will the tool allow them to edit this with minimal development assistance? Or is it more targeted towards changing text and images? It’s also good to keep in mind what kind of experiments you will want to use in the future. This can be hard to anticipate, but keeping this in mind, it’s good to find a tool that has the flexibility to test multiple different kinds of variables in different settings.

How easy is it to use? Will you be able to set up an A/B test in minutes, or will it take hours of work? This is a very important factor to consider, especially considering the resources and expertise you have at your disposal. If you’re a small company or startup, you probably want to start with something more on the simple side.

How easy is it to integrate with your website? Does this tool require technical knowledge that is above your skill level? Again, if you have developers to help you implement your A/B testing tool, that’s great. You can customize your setup to meet the exact needs of your organization. But if you don’t have the luxury of having developers at your disposal, look for something easy with low or no-code required to set up.

What kind of customer support does this tool offer? Do they have a live chat feature or simply email responses only? If something goes wrong with your A/B test and there’s no one you can talk to immediately about fixing it, then there might not be much point in using this solution at all. There’s nothing more frustrating than something going wrong with your experiment data and having nobody to respond to you when you need them most!
Finding the right tool for the job is crucial to get the most out of the tests you want to run. It can be hard to choose the right tool with so many options on the market, but as long as you consider the 4 points above you should be sufficiently informed to make a solid choice for your website.

4. Setting up an A/B test

Setting up an A/B test can be simple, but it very much depends on what you want to test, and what tools you have available. If you want to test things like changing the colour of an element or the text on a page, most no/low-code tools can help you with this. But if you want to change more complex things like the navigation structure of your site you might need a developer to help you. Once you've decided on your goals, chosen the best metric and found the right time to test, you'll be ready to get started.

The first step is creating two versions of a webpage—the control and the experimental variation. For example, if you're testing whether or not your messaging has improved conversion rates in comparison to the original version of your homepage, then you would create two different homepages with different messaging concepts displayed on them: one with "Free Shipping" as its headline; another with "Savings." Your goal would be for visitors who see each page (control group vs variant group) to convert into customers at similar rates as those who previously visited this page without any changes made (i.e., no free shipping vs no savings).

In a simple A/B test, you would create a 50/50 split of all traffic to the page you are testing. However you can also tweak the exact percentage with certain tools such as Google Optimize. Just keep in mind that you don’t want to make an experiment group too small, as you will still need a large enough data sample. You will then need to measure how many people from each group converted into customers after visiting either version of this page. Based on the findings from this measurement, determine which one works better than others when it comes down to converting users into paying subscribers.

5. Interpret your results

Let’s say you have a hypothesis of “the new red button will increase conversions by 15% relative to the existing conversion rate.” You run an A/B test and get the following results:

• Control (original page layout) — 20% conversion rate
• Intervention (red button) — 26% conversion rate

Relative to the conversion from the original layout, the intervention performed 33% better! This means that your hypothesis was correct, so you should ensure this red button is published on your live website for all visitors. If your hypothesis was incorrect, you would need to go back to the drawing board and try to analyse where you are losing your visitors and why.

Keep in mind that there are a lot of potential extraneous variables that can affect the results of your A/B test. You can’t always control who goes to your website and why. This is why generally we encourage you to try and use as large a sample as possible in your A/B tests. This way you can start to identify trends and have a higher degree of confidence in the validity of your results.

Pro tip: Optimize visual content with attention prediction

Sometimes it can be hard to know why certain changes have an effect on your conversion rates. Some things just seem to ‘work’ (or not). But one great way to predict better conversion is by predicting visual attention.

Attention prediction is a relatively new conversion optimisation technique that uses computer vision and machine learning to measure attention and engagement. Using attention prediction as part of your A/B testing strategy allows you to optimise visual content to improve conversion rates, engagement, and even customer satisfaction.

Traditional A/B testing can be difficult to implement without developer assistance. And most of all, it’s extremely time-consuming. To be able to gather any meaningful insights from your A/B test, you need to let it run on your live website for 2-3 weeks at the very least. You also need to ensure you have hundreds (if not, thousands) of regular website visitors. Any less than this and you might not obtain a representative data sample. But with attention prediction, both of these limitations are removed.

No technical setup is required - all the analysis happens within your web browser or from a plugin within your preferred design software. And most importantly, no participants! So no need to wait for weeks to get your results, simply upload a screenshot of the page you wish to optimise (or use the Chrome plugin), and get your attention prediction instantly to see which visual elements of your website grab the most attention.

Take a look at the example below, which compares two different approaches to a website home page. The CTA button in version A receives 4.7% of attention, compared to only 1% for version B. Just imagine the amount of potential conversions that would have been lost if you had to run an experiment for a month to test the different options.

So, now you have everything you need to know about setting up an A/B test. But perhaps you're still wondering what's the best way to create your website? If you want a professional and responsive website, we recommend WebWave drag and drop website builder. Their product allows even absolute beginners to create websites. You will be able to quickly create a portfolio, landing page, small online store or any other type of website. You can choose from ready-made free templates or start from scratch.

‍