> For the complete documentation index, see [llms.txt](https://docs.mediarithmics.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.mediarithmics.io/measurement-and-attribution/measure-and-optimize/set-up-an-a-b-test/read-and-interpret-your-experiment-results.md).

# Read and interpret your experiment results

Check that your experiment computed correctly, that its groups are balanced, and why some users were left out.

## Your goal

Trust, or catch a problem with, the groups an experiment produced, before you activate a campaign against them or draw a conclusion from comparing them.

## Expected outcome

A clear read on whether the groups are balanced and large enough to measure on, or a diagnosed reason why they are not.

## Find your experiment

The **Experiments** menu, located in the **Measurement** section, lets you view and manage all experiments created on your datamart. It displays a customizable table where the experiment name is shown by default. Clicking a name opens the experiment details.

## Compute it, and follow its status

The experiment details page is displayed after you click **Create Experiment**. At that stage the experiment has not been computed and is marked **Draft**. To run it, click **Compute**.

<table data-search="false"><thead><tr><th width="160">Status</th><th></th></tr></thead><tbody><tr><td><strong>Draft</strong></td><td>The experiment has not yet been launched and can be edited.</td></tr><tr><td><strong>In progress</strong></td><td>The randomization is being computed. It can no longer be edited.</td></tr><tr><td><strong>Uploading</strong></td><td>The randomization has ended and the groups are being populated.</td></tr><tr><td><strong>Completed</strong></td><td>The groups are populated.</td></tr><tr><td><strong>Error</strong></td><td>Something went wrong. The experiment is not recorded and can be edited. Contact your CSM or the Support team.</td></tr></tbody></table>

{% hint style="info" %}
Groups, as segments, are created as soon as **Compute** is clicked. Although they are not populated at that stage, as long as the status is not **Completed**, feed configuration is already possible.
{% endhint %}

## Read the results

The results of each experiment are detailed in two tabs.

### Overview

{% hint style="info" %}
This tab is only available for experiments created using **stratified randomization**.
{% endhint %}

<table data-search="false"><thead><tr><th width="240">Section</th><th></th></tr></thead><tbody><tr><td><strong>Groups details</strong></td><td>Group size, both in number of UserPoint and in number of identifiers available, according to the identifiers selected in the creation form.</td></tr><tr><td><strong>Representativeness details</strong></td><td>A synthetic view of the randomization performed, used to validate group balance. Each criterion is analyzed through 2 graphs.</td></tr><tr><td><strong>Strata details</strong></td><td>Detailed results of the randomization: the strata created, and how each was distributed across the groups.</td></tr><tr><td><strong>Unusable UserPoint</strong></td><td>The UserPoint of the source segment that were not distributed to any group, and why.</td></tr></tbody></table>

#### Reading the representativeness graphs

* The first graph, on the left, shows the distribution of UserPoint in your segment. It is for information only and says nothing about the randomization performed.
* The second graph, on the right, shows how the segment's UserPoint were distributed between the groups. **A histogram showing similar bars is proof that randomization went well** and the groups are balanced.

{% hint style="info" %}
If you run an experiment with 3 representativeness criteria, this section displays 3 x 2 = **6 graphs**.
{% endhint %}

#### Reading the strata details

If a stratum is too small to ensure balanced group distribution, it is excluded from the experiment entirely, and strata excluded for that reason are **highlighted in red**. The threshold is the [acceptance threshold](/measurement-and-attribution/measure-and-optimize/set-up-an-a-b-test/create-your-test-and-control-groups.md#acceptance-threshold).

If a significant portion of UserPoint are eliminated because of small strata, it may indicate that:

* the segment used for the experiment is too small, or
* the experiment includes too many strata, which can result from using too many representativeness criteria, or criteria with too many distinct values.

### Groups

This tab is dedicated to the segments created for the experiment. As many segments are created as there are groups configured in the creation form.

The name of each segment is established by default from the name of the experiment and a prefix of the form *\[group\_type]\[test group number]\[size\_expressed\_as\_a\_percentage]*, with:

* \[group\_type] = "Test" or "Control"
* \[test group number] = "1", "2", and so on
* \[size\_expressed\_as\_a\_percentage] = "90", "10", and so on

{% hint style="info" %}
An experiment created with 2 test groups, each comprising 45% of the segment, and a control group comprising the remaining 10%, has its segments named as follows:

* Test\_1\_45\_*experiment\_name*
* Test\_2\_45\_*experiment\_name*
* Control\_10\_*experiment\_name*
  {% endhint %}

All segments created from experiments can be identified from the Segments menu using the `Segment type = Experiment groups` filter.

## Why the groups hold fewer users than the source segment

It is very common, and completely normal, for a portion of the original segment's UserPoint to be excluded during distribution. Three filters cause it.

<table data-search="false"><thead><tr><th width="240">Reason</th><th></th></tr></thead><tbody><tr><td><strong>The UserPoint has no expected identifier</strong></td><td>If you are planning an email campaign, for which you need an email address to send your message, but the segment you are working on is made up of UserPoint who do not have an email, then those have to be eliminated before computing the randomization, in order to guarantee both group size and balance. The identifiers concerned are the ones selected in the <em>Addressability</em> section.</td></tr><tr><td><strong>The UserPoint is an outlier</strong></td><td>Stratified randomization only. All UserPoint classified as outliers on at least one representativeness criterion are eliminated before the randomization is computed. Outliers are calculated via the Z-score and configured in the Computing Console. See <a href="/measurement-and-attribution/measure-and-optimize/set-up-an-a-b-test/create-your-test-and-control-groups.md#outliers">Outliers</a>.</td></tr><tr><td><strong>The UserPoint belongs to a stratum too small</strong></td><td>Stratified randomization only. All UserPoint belonging to a stratum too small to guarantee group balance are eliminated before the randomization is computed. See <a href="/measurement-and-attribution/measure-and-optimize/set-up-an-a-b-test/create-your-test-and-control-groups.md#acceptance-threshold">Acceptance threshold</a>.</td></tr></tbody></table>

{% hint style="success" %}
**Best practices for limiting the number of UserPoint excluded from an experiment:**

* When creating a segment on which you will run an experiment, add a condition to keep only those UserPoint with the identifier that will be used for activation.
* Ensure that the number of representativeness criteria is consistent with the size of your segment.
  {% endhint %}

## Why a group size still moves afterwards

From a UserPoint point of view, the list assigned to a group by stratified randomization is **set in stone** at the time of calculation: the experiment does not redraw profiles.

You may still observe slight fluctuations at segment level, or during activation through your feeds, for two reasons related to the identity engine.

<table data-search="false"><thead><tr><th width="240">Phenomenon</th><th></th></tr></thead><tbody><tr><td><strong>Graph ID merges</strong></td><td>Over time, different UserPoint may be merged if they are recognized as belonging to the same person. This leads to the technical deletion of obsolete UserPoint, some of which may belong to your experiment groups, causing the segment size to decrease slightly.</td></tr><tr><td><strong>Identifier updates in feeds</strong></td><td>The identifiers sent to your activation partners evolve at Graph ID level. Even though a UserPoint remains fixed within its group, the email addresses associated with it can be added, modified, or deleted over time. This is why you may see additions and deletions flowing through your feed.</td></tr></tbody></table>

## Verify

* Every group segment exists and, once the status is **Completed**, holds users.
* For a stratified experiment, the two representativeness graphs per criterion show comparable bars across groups.
* No stratum you expected to matter was silently excluded. Check the strata details if a criterion's balance looks off.
* The delta between the segment size and the sum of the group sizes is explained by the three filters above, and not by something unexpected.

## Related pages

<table data-view="cards" data-search="false"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Create your test and control groups</strong></td><td>Adjusting the configuration that produced these results.</td><td><a href="/measurement-and-attribution/measure-and-optimize/set-up-an-a-b-test/create-your-test-and-control-groups.md">Create your test and control groups</a></td></tr><tr><td><strong>Set up an A/B test</strong></td><td>What an experiment is, and why incrementality needs a control group.</td><td><a href="/measurement-and-attribution/measure-and-optimize/set-up-an-a-b-test.md">Set up an A/B test</a></td></tr></tbody></table>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.mediarithmics.io/measurement-and-attribution/measure-and-optimize/set-up-an-a-b-test/read-and-interpret-your-experiment-results.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.