A/B testing. Improve and personalize the User Experience – | Blog

What is an A/B Test

A A/B testing It basically consists of comparing two versions of the same page within a website or application to obtain results of which of the two versions improves the main objective of the page (KPI).

The way this test works, in general terms, consists of dividing the traffic received by the page and diverting a part to variant A and the rest to variant B.

There are different tools to perform A/B testing. Thanks to them, we obtain a series of data, metrics and statistics of each of the variants that help us determine which one obtains a better performance and increases the conversion rate.

Among other objectives, you can check which variant generates more clicks, subscriptions, carts started, sales, page views, etc.

The websites where A/B tests are most often carried out are those whose main objectives include lead generation, editorial media or E-commerce.

Test Types

There are different types of tests or tests among the most prominent are the following:

A/B test

Its about A/B testingalso called A/B/n testing. It is a random experiment that uses two or more variants of the same web page. The original variant (A) is always used. Variants B through n contain one or more elements that have been modified from the original version.

This type of test is usually the most used. It is often used, for example, when we want to change the color or copy of a button and measure how many clicks each variant gets.

redirect test

In this type of test, different URLs are used for each version that help to test two independent web pages. It is frequently used for when a redesign is carried out and both the structure and the is completely new with respect to the original version.

Multivariate test

This type of test is performed when several elements are modified on the same web page simultaneously to check which composition is the most effective and yields the best result.

This type of test measures the impact of each of the elements that have been modified and how they have affected the previously defined objective.

How to do an A/B Test

Before launching an A/B Test, it is necessary to plan a series of aspects:

1. Define the goal

You have to be clear about what you want to analyze. For example: making a change to a call to action (CTA). We can change the color of a button or the copy it contains from “Add to cart” to “I want it!”.

In summary, it is about being clear about the function of the page that you want to optimize (increase leads, achieve greater visibility, increase sales, etc.).

2. Analyze and define the variants of the experiment

We will make the specific changes to the original page generating as many variants as necessary that will be compared with the original page or variant A.

It is at this point where we change the copy, design, calls to action or CTA, headlines, etc.

3. Weighting of variants

It consists of defining the proportion of traffic from target visitors that goes to each variant. The most logical thing is that if we have the original variant (A) and a variant B, the traffic is segmented into 50% for A and 50% for B if we want to carry out the experiment impacting 100% of the traffic that the page receives. which we are going to test.

If you want to weight 100% of the traffic to variant B and not use the original version to carry out the experiment, you can weight 100% to variant B and 0% to the original variant (A).

4. KPIs and segmentation rules (Who and When)

It is about declaring measurable and quantifiable objectives such as the number of page views, baskets started, clicks, session duration, bounce rate, etc.

At this point you have to define the segmentation rules, it consists of:

• Define the geographical scope of the target visitors to whom the experiment will be launched. It can be a country or a specific city.
• Define the origin of visitors such as those from Twitter, Facebook, only organic traffic, etc.
• Define the technology in which the experiment will be carried out. It can be on mobile devices, tablets, PCs, in certain browsers or in a specific operating system.
• duration of the experiment.

The experiment must be active until at least one of the following conditions is met:

• That 15 days have elapsed, to be able to check the variations in traffic for a whole week.
• At least one variant must have a 95% probability of being better than the rest of the variants or the reference value.

Having defined the KPIs and the segmentation rules, we can launch our experiment.

After the completion of A/B testing A report must be made that collects the data obtained and the conclusions.

Important data for the A/B Test report

When making the report, a series of important data must be included so that the people who are going to view it understand and contextualize the experiment that has been carried out.

General report details:

• Goal.
• Variants used in the experiment.
• Weighing.
• KPIs and segmentation rules.
• duration of the experiment.
• Description of the experiment.

Specific report details:

• Total sessions of each day of execution of the experiment by variant.
• Overview of the improvement for each variant with respect to the total number of sessions.
• Probability of being the best variant.
• Probability of outperforming the original per variant.
• Conversion rate according to KPIs per variant.
• Number of conversions achieved per variant.
• Conversion rate according to the KPIs defined over time by variant.

There are numerous A/B testing tools. google offers Google Optimize for free.

credits