For the complete documentation index, see llms.txt. This page is also available as Markdown.

Set up an A/B test

Split an audience into test and control groups to measure a campaign's real, incremental effect.

Your goal

Find out how much of an outcome your campaign actually caused: not just how many exposed users converted, but how many would not have converted without it.

Expected outcome

A set of segments, one or more test groups and usually a control group that is never exposed, balanced well enough that the difference in their conversion rates can be attributed to the campaign rather than to chance.

What an experiment is

Experiments makes it possible to carry out A/B tests to run incremental measurements. Incrementality is the measure of the additional impact, or lift, generated by an action beyond what would have happened without it. In the context of ads, it is the measurement of ad effectiveness.

Running an incremental measurement involves creating different groups from the initial audience you want to target for your campaign, at least one of which will not be exposed to the campaign. That unexposed group is the control group.

Depending on the objective of your campaign, you may want to create groups with a particular focus on certain criteria likely to influence conversion, so that your groups are fairly distributed on those key criteria.

If the aim of your campaign is to generate sales of a particular product, and you know that age is highly likely to influence purchase, you want to be sure your groups are balanced on age to guarantee the reliability of the measurement you carry out later.

Balancing groups here means ensuring each group contains the same proportion of 18-24 year-olds, the same proportion of 25-34 year-olds, and so on.

Experiments speeds up incremental measurement projects by letting you create groups and manage their balance in a few clicks.

Create one from a segment

To create an experiment from a segment page, click the More button in the top right corner, then select Create Experiment.

Manage every experiment created on your datamart from the Experiments menu, located in the Measurement section. It displays a customizable table where the experiment name is shown by default. Clicking a name opens the experiment details.

How the job breaks down

The segments an experiment produces, and what simple randomisation draws from, are documented on Experiment groups segment and Partition segment.

Last updated

Was this helpful?