Detailed Guide On A/B And Multivariate Testing With Case Study

Testing is a very good way to understand what your website’s visitor may want from you. Through the right testing plan, you have a clear idea about what the users like in your website.

A/B testing and Multivariate testing are not a very common topic in the field of online marketing. Most of the marketers are still not aware about it. They don’t know what is it and how to implement it in the online marketing plan. Thus, to help with your marketing plan, here is the best guide for A/B testing and multivariate testing.

A/B Testing

Before you plan A/B testing, firstly you must know “what is A/B testing” and “how it can be helpful to your website”.

.As the name sounds, In A/B testing there are two versions of the same web page and you need to test the pages to find which one is performing better.

A B Testing

More specifically, in A/B testing, your website has two design pages A and B where A is the existing page design and B is the is the newer page that you want to test. Both pages gets the equal amount of traffic and you measure the performance on the basis of your goals such as sales, conversion rate etc. Finally, at the end of testing, you analyze the results to find which version has performed better.

How to Plan for A/B testing

To plan A/B testing, firstly you need to decide what to test. The choice of test fully depends on your goal. For example, if you want to increase the number of sign ups, then you may need to test the types of fields in the form, number of fields in the form etc. During the testing, you need to analyze the core issue that is preventing the visitors from signing up.

On the other side, if you want to increase the number of sales, the testing elements may be the location of call to actions, text in call to action, color of buttons etc. However, the elements may differ for each testing, but there are certain elements that are commonly tested such as

• Size, color, text of call to actions
• Number of fields in form
• Images on the landing page
• Headlines and sub-headlines on the page
• Length of text

Before you start A/B testing, clearly understand your goal and carefully decide the elements that you need to test. Remember, that the testing of both the pages is performed simultaneously. You can’t test one page today and next page tomorrow.

How to perform your First A/B test

Basically, the whole process of A/B testing has 3 easy steps

• Choose your goal – As I have mentioned earlier that before you plan A/B testing, there should be a clear and straightforward goal in your mind. You should have a clear idea about the results that you are looking for.

• Create a different version of the page and run the test – Based on what element you want to change, create a different version of the page. Once you create the alternate version, you can run the test and track the live status.

• Find the Winner page – After, you successfully run the test, you will have the winner. Now, you can replace the page with a new one that has performed better.

How casa mineira has increased the leads by 57.25% through A/B testing (Case Study)

Casa Mineira is a leading real estate company in Brazil. The original homepage of the website was having a simple search box to find properties. To increase the conversion, they created a different version of the home page which consist of two search box, first one with a drop down menu allowing the users to select the type of property and the second one to choose the location. This clear choice for searching the property has outperformed and increased the business leads by 57.25%

Multivariate Testing

When you want to expand the testing process to a deeper level, then Multivariate testing is the best choice. Where A/B testing, test a page for one element, Multivariate testing, test the page for more than one element with different combinations to find which one is yielding higher conversion rate and better results.

Multivariate Testing

A/B testing is simply a choice between two, means only two variations are presented to the users and finally one is selected, but multivariate testing allows to test a number of variations at the same time.

In multivariate testing, the web page is fragmented into small units and then multiple variations are created based on these small units. Here, the small units mean the elements that you want to test in multivariate testing. For example, if you want to test the page for the headline and call to action buttons, then you probably will have the following options

Headline 1 and Headline 2
CTA 1 and CTA 2

Now there will be four combinations to create web pages and the traffic is splitted between these 4 pages to find which one is performing better.

1. Headline 1 and CTA 1
2. Headline 1 and CTA 2
3. Headline 2 and CTA 1
4. Headline 2 and CTA 2

To better understand the multivariate testing, let’s take an example. Suppose, you want to perform multivariate testing for an email marketing campaign. The testing plan will have the following steps.

1. Before you run the test, firstly check your previous mail copies and find which elements can be improved. The email may have the elements like subject, headings, subheadings, banners, etc that can affect the conversion rate.

2. When you find the elements that need to be tested, create the variations and set up a testing schedule.

3. When you are ready for the test, run the email marketing program. Send the email variations to the different groups of people and check which variations is getting more number of clicks and generating more lead.

How VRTP lifted the overall order conversion rate by 7% using multivariate testing (Case Study)

VRTP is Village Roadshow Theme Park, the largest theme park in Australia, providing entertainment, rides and slides. They implemented multivariate testing plan to achieve higher conversion rate and revenue. They created the different versions of ticket options for VIP and non VIP guests. After the successful testing, they found that each new variation has increased their add to cart rates, which resulted in 7% lift in order conversion rate.

Instead of doing worthless experiments on your website to find out the reasons of not getting the higher conversion rate, choose a right testing strategy to understand the behavior of visitors. Slight changes in color, text, buttons or design may have a startling impact on your website. Thus, to achieve the best results, continuous testing is very important.