Case Study: Launching Complex A/B Tests for Nuclear Blast

scandiweb’s CRO and UX specialists wanted to do a complete checkout redesign for our client Nuclear Blast and involved the Analytics team in the process due to the rather complex A/B testing it requires.

Challenge

At first, it seemed doable. We thought we could bring Figma designs to life via Google Optimize and multiple Javascript, HTML, and CSS changes. Unfortunately, we quickly realized Google Optimize can be limiting for complex A/B tests. 

Google Optimize is an all-in-one tool that allows the business owner, marketing specialist, or web developer to apply site-wide or targeted changes to a site, split users into experiment groups, apply UI/UX changes and collect data when desired goals and events have occurred. Because it is an all-in-one tool, some features can become very limiting, like being able to redesign an entire page from the ground up and display it to the end-user without slowing down the site and hurting the UX.

Our approach

To solve this challenge, we decided to split the work into 2 parts:

• Variant redesign (developers) 
• Experiment application and data collection (Analytics team)

We wanted to keep using Google Optimize as the brains of the operation for its effective audience splitting capabilities, easy-to-use data collection hub, and compatibility with Google Analytics, and we wanted to have the experiment variant set in code by web developers. 

The challenge now was to find a way to tell the site when to apply one variant or another. Our solution to this riddle was fairly simple—to use the _gaexp cookie Google Optimize assigns users when they come to the site or visit a specific page. 

Google Optimize is a script that can be installed synchronously (blocking) or asynchronously (non-blocking). If you launch an experiment for that particular Google Optimize ID, a cookie will be set. Depending on the cookie value, the user can be assigned the default look and feel of the site or the experiment variant. 

For example, if a user comes to the site, they will be assigned the _gaexp cookie with a value GAX1.2.SPxpr4GySp7joljBIaiYiQ.19208.0. Because the cookie ends in .0, we know that Google Optimize has assigned the user the default variant, and no changes should be applied.

If the user receives the _gaexp cookie that ends in .1, we know that Google Optimize has assigned the user the first experiment variant, and the created variant should be applied to the site. 

Result

This approach allows us to not only run complex A/B tests reliably but also divide and conquer the workload among teammates who specialize in different things. Web developers can work on site redesigns and implement the logic for reading the _gaexp cookie, while Analytics specialists configure and launch the experiment, collect data, and analyze the results.

Most importantly, everyone avoids having to do the work twice! If the experiment hypothesis proves successful, the underlying code is already written. Implementing the variant and making it the new default will be easier than having to base your work on designs that would have come from Google Optimize.

Found this solution useful? Need assistance setting up A/B tests on your website? Shoot us a message at [email protected], and our CRO and Analytics teams will get back to you within 48h.

Share on: