Skip to main content

CUPED

Controlled-experiment Using Pre-Experiment Data (CUPED) uses data on your existing visitors to accelerate reliable experimentation and personalization campaign results.

Key benefits​

CUPED allows you to make data-driven decisions faster and with more confidence, provides better estimates for the effect of variations, and reduces the risk of false positive results for tests with fewer data points.

When should you use CUPED​

CUPED will be most effective if:

  1. You have been collecting data on your goals prior to the launch of the experiment, for at least two weeks.
  2. There is correlation between goal conversions before and during your experiment.

Additionally, you should consider your experiment's goals before enabling CUPED. We recommend you activate CUPED for the main metrics of your site, such as transactions.

Enable CUPED​

To enable CUPED:

  1. Navigate to the Data tab in the right-hand sidebar

  1. Under Advanced settings, click the toggle next to CUPED.

  1. Click the dropdown menu, then check the goals for which you want to enable CUPED.
  2. Click Apply.
note

Data loading time may take slightly longer than for the classic algorithm. We advise limiting CUPED to your key goals.

The data presented on the page will change depending on the goals selected in the pop-in.

If you selected your primary goal from the drop-down, the main results block will present CUPED-adjusted data, and it will labeled CUPED enabled.

Similarly, the results tables for each of your goals will be impacted, according to the goals for which you enabled CUPED.

You’ll notice three differences in your results columns:

  • Conversion rate becomes Conversion rate adjusted
  • Improvement rate becomes Improvement rate adjusted
  • Reliability becomes Reliability adjusted

Note that you can apply filter-based breakdowns to your data in the results tables only if you disable CUPED. When you click Breakdown on a table, you must confirm that you want to disable CUPED. You can reactivate CUPED after removing the breakdown filter.

note

CUPED gets more effective over time. The more you use Kameleoon, the more experiment data you’ll have that our algorithm can use to improve the effectiveness of CUPED.

Manage CUPED​

To modify your CUPED configuration, open the Actions menu and select Manage CUPED.

You can select and deselect goals in the pop-in window. Click Validate to save your changes.

Disable CUPED​

To revert to the classic statistical method, open the Actions menu and select Manage CUPED.

You can deselect your goals in the pop-in, and click Validate. After saving your changes, CUPED will no longer be applied to your results. 

note

Once you enable CUPED, you must rely on its results. Switching between the original and CUPED-adjusted results and making decisions based on the more favorable outcomes will increase your probability of overestimating the lift of your experiment and risk of getting false positive results.

You can read more about CUPED in our statistical paper.