A/B testing a new landing page
Question
Suppose you are working for an e-commerce company and the marketing team is trying to decide if they should launch a new webpage. They ran an A/B test and need help analyzing the results. They provided you with this dataset, which contains the following fields:
- user_id: the user_id of the person visiting the website
- timestamp: the time in which the user visited the website
- group: treatment vs control, treatment saw the new landing page, control saw the old landing page
- landing_page: new vs old landing page, labeled 'new_page'/'old_page'
- converted: 0/1 flag denoted whether the user visiting the page ended up converting
Given this information, you're asked to come up with a recommendation for the marketing team -- should the marketing team adopt the new landing page? The team wants the landing page with the highest conversion rate.
Note: the control group should always align with the old landing page, and treatment group should always align with the new landing page. If these do not line up, you should drop the rows before doing the analysis.
To help get you started, the code below loads in the dataset. You can also make a copy of this Colab notebook to get started!
# Importing packages
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
# Reading in data
df = pd.read_csv('https://github.com/erood/interviewqs.com_code_snippets/blob/master/Datasets/ab_data.csv?raw=true', parse_dates=True)
df.head()
| user_id | timestamp | group | landing_page | converted | |
|---|---|---|---|---|---|
| 0 | 851104 | 2017-01-21 22:11:48.556739 | control | old_page | 0 |
| 1 | 804228 | 2017-01-12 08:01:45.159739 | control | old_page | 0 |
| 2 | 661590 | 2017-01-11 16:55:06.154213 | treatment | new_page | 0 |
| 3 | 853541 | 2017-01-08 18:28:03.143765 | treatment | new_page | 0 |
| 4 | 864975 | 2017-01-21 01:52:26.210827 | control | old_page | 1 |