A/B testing a new landing page


Suppose you are working for an e-commerce company and the marketing team is trying to decide if they should launch a new webpage. They ran an A/B test and need help analyzing the results. They provided you with this dataset, which contains the following fields:

  • user_id: the user_id of the person visiting the website
  • timestamp: the time in which the user visited the website
  • group: treatment vs control, treatment saw the new landing page, control saw the old landing page
  • landing_page: new vs old landing page, labeled 'new_page'/'old_page'
  • converted: 0/1 flag denoted whether the user visiting the page ended up converting

Given this information, you're asked to come up with a recommendation for the marketing team -- should the marketing team adopt the new landing page? The team wants the landing page with the highest conversion rate.

Note: the control group should always align with the old landing page, and treatment group should always align with the new landing page. If these do not line up, you should drop the rows before doing the analysis.

To help get you started, the code below loads in the dataset. You can also make a copy of this Colab notebook to get started!

# Importing packages
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
# Reading in data
df = pd.read_csv('https://github.com/erood/interviewqs.com_code_snippets/blob/master/Datasets/ab_data.csv?raw=true', parse_dates=True) 
user_id timestamp group landing_page converted
0 851104 2017-01-21 22:11:48.556739 control old_page 0
1 804228 2017-01-12 08:01:45.159739 control old_page 0
2 661590 2017-01-11 16:55:06.154213 treatment new_page 0
3 853541 2017-01-08 18:28:03.143765 treatment new_page 0
4 864975 2017-01-21 01:52:26.210827 control old_page 1


Access restricted

Subscribe to premium account to see the solution.

Get premium now