Quick Guide to use Google Optimize to run an AB Test

Hi, this is Jing, a data scientist with great passion for applying data science and big data technology in the industry. Today you are going to learn about Google Optimise to try your first AB test with me.

Clarification on AB Test and Web experiment

AB Test is also known as split testing, is a method of comparing two versions of a webpage or app against each other to determine which one performs better. It is just one type of web experiment, there are other types, such as split url test, multipage test and etc.. depends on different categorisation. But usually we call web experiments as AB test in general.

Google Optimize – Client Side AB Testing

Google Optimize

Google Optimize, It is a free testing tool. Optimize 360 is a paying version of Google Optimize. The advantage of upgrading Optimize to Optimize 360 is, we will not have a limitation of 16 combination during one experiment and could run more than 5 experiments at a time.

How does the test work by Google Optimize

In the standard implementation of Google Optimise test, the browser executes JavaScrip in the web page to determine which variant to serve. These are the client-side tests.

Client and Server Side in a web experiment

Supported Types of Tests in Google Optimize

A/B Testing

This is an experiment in which two variations of the same page are tested to know which performed better in providing the best user experience. Here, Variation A is the original version, and Variation B is a variant of that page, with one or more elements modified.

Split URL Testing

In this test, the two different web pages compete against each other, and variations are defined by the URLs instead of by modified page elements. This type of test is often used when the variants are two different landing pages with a full page redesign.

Multivariate Testing

The variants include multiple combinations of elements to help find out which one creates more engagement. If there is one element has been changed, then it will be an AB test with two variants in total. If there are X elements has been changed, then it will be 2^x variants in this experiment. Recommend this for smaller changes on one page such as content copy changes. So that the impact of these changes can be amplified.

Use Cases for using Google Optimize

Most tests implemented in the Google Optimize are limited to client-side changes, such as changing the style or position of an element on a rendered web page. These changes can be easily made in the Visual Editor Google Optimize provides. Here are some users cases which you can use Optimize to test without Dev support.

1. Copy changes on the existing component on the web page
2. CTA link change
   • Text of the CTA or the destination of the CTA
3. Position of an existing component
   • Change the order of the component or remove certain component
4. Change an image/ styling of an existing component
5. A new simple component just with images, links or texts (no functions)
   • Require certain HTML knowledge, but it is possible
6. To be added …

Setup Requirement when using Google Optimize

If your tests are one of the use cases above, then there is no dev support needed for the test. As long as you have an Editor Access to Google Optimize, you could setup a test on your own.

Tips for setup test activation event

• Most of our webpages are built dynamically, so you need to change ‘Activation Event’ into ‘Continuously’ so that you could see your variants rendered properly in the Preview Mode.
• Theoretically, you could custmize the Activation event, but I don’t know how right now.

Activation Event Setting in Google Optimize

Common Problems & Limitations with Google Optimise

Flickering Issue while testing at client side

Flickering is when original content appears to a user before an experimental version showing during testing. It is a common problem for most testing tool when implementing a client side test. This happens because of an asynchronous script with the test tool. Meanwhile, this asynchronous makes the test page not be slowed down due to the test.

Page Load Mechanism while testing in client side

Solution of Testing Flickering Issue

However, the flickering issue only becomes a problem which will effect the test result significantly when the test is above the folder.

• For the test whose changes are below the fold, flickering will not be a problem since user will not be able to notice the flickering. So, just do the test.
• For the test whose changes are above the fold, flickering issue needs to be tackled before implementing the test.
  1. Either switching to a server side implementation such as through LaunchDarkly
  2. Implement an anti-flicking snippet to reduce the flicker to 0.1 ms so users won’t see anything.
  3. Another solution is changing the way of loading page:
     • For example, sending a simple (fairly empty) page that has allocated the entire page space but without any content on it. By doing so, the first “update” is always coming from the client. As a result, testing on such as page above the folder using client side will not lead any flickering issues. The consequence is, we pay with a slower FCP.

Preview Function Issue in Google Optimise

Sometimes the preview function will stop working. There will be a message saying ‘Make sure you enable cookies and restart your browser’. The reminding message can be misleading. It does not fix the problem. What you can try is:

1. Turn off the ‘Preview’
2. Go to the ‘Debug’ mode to check the changes
3. Make sure there is no bugs
4. Reopen the ‘Preview’ mode again

Measurement & Analysis in Google Optimise

Measurement of the performance of each variant is done by setting objectives in the Google Optimize. The analysis of which variant is a leader for each objective is automatically done by Google Optimize. There is a dashboard monitoring the performance of each variant for each objective you choose and showing you the analysis result of the statistical model indicating which variant is the leader daily.

Objective setup for web tests

There are three types of objectives you can choose in Google Optimise

• System objective
  • Bounces (Exit)
  • Pageviews
  • Session Duration
  • …
• Custom Objectives
  • Pageview type
  • Event type
• Analytics Goals
  • Goals from linked Analytics view are imported into Optimize for use as experiment objectives.

Within Optimize 360, up to 10 Objectives per experiment can be set. System Objective and Analytics goals can be added after the experiment has started. More details could go to the help page of Google Optimize.

End

All in all, I will say Google Optimize a very easy tool to kick off your first AB test. And if you are testing a lot of visual changes, Google Optimize is definitely one of your best options to run the test. Hope this can be helpful to you. Thanks for reading. I am Jing, a data scientist aiming to be better and better.

Advertisement