> For the complete documentation index, see [llms.txt](https://docs.adpage.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.adpage.io/data-and-insights/data-studio/run-an-experiment.md).

# Run an experiment

Run a regional test in Experiments: read the proposal, check the regions, start it, keep the logbook and fetch the result. Also covers how Black Friday and other holidays are handled.

Your Marketing Mix Model estimates what each channel delivers, and the ad platforms report what they claim. **Experiments** checks both with a test: switch a channel off in a few provinces (a holdout), or add budget there (a scale-up), and compare those provinces with the rest of the country. The difference is what the channel really delivers.

{% hint style="info" %}
**Experiments is part of Marketing Mix Modelling.** You find it in the menu under **Insights**. Not there? Ask <support@adpage.io>.
{% endhint %}

The screenshots on this page show a made-up store, with the app in Dutch.

## How a regional test measures

* AdPage adds up the weekly revenue of the test provinces, using the shipping postcode of each order, across all connected stores.
* From the weeks before the test, it learns how the test provinces move along with the control provinces: a weighted mix of the control provinces that follows the test provinces as closely as possible.
* During the test it carries that mix forward. The gap between what the test provinces did and what the mix predicts is the effect of the channel. Divided by the budget you moved, that gives the incremental ROAS (iROAS).
* Only whole weeks, Monday to Sunday, count.
* The test measures revenue, not the number of orders. If a few very large orders dominate small provinces, AdPage can set a maximum per order (for example € 500): every order then counts up to that amount.

## Before you start

* **Marketing Mix Modelling is set up for your container**, with your webshop and your ad accounts connected.
* **Enough history.** A test can last at most a third of the history before it: a 10-week test needs at least 30 weeks. With a year or more, last year's holidays are in it too.
* **One test at a time.** A second test changes the account during the first and spoils both.
* **Regional tests are for Google and Meta.** Other channels are tested nationally.

## Run an experiment

{% stepper %}
{% step %}

### Open Experiments

Go to **Experiments**. Each tile is one experiment, with its phase: **Proposal**, **Runbook**, **Measuring** or **Outcome**. Use the tabs to show one phase, and select a tile to open it.

<figure><img src="https://data.adpage.io/help/experimenten/01-overzicht.png" alt="The Experiments page with a tile per experiment"><figcaption><p>Every experiment of the container, with its phase</p></figcaption></figure>
{% endstep %}

{% step %}

### Read the proposal

A proposal answers four things:

* **The question**, for example *Does Meta pay for itself?*
* **What we do**: the provinces, and what happens to the budget there per week.
* **Can the test see it?**: the lowest iROAS the test can tell apart from noise, next to what the platform and the model say. If the platform's own claim is below that line, no outcome would mean anything.
* **What you will know** after the test, and what you do then.

<figure><img src="https://data.adpage.io/help/experimenten/02-voorstel.png" alt="A proposal card with the question, the regions and the detection threshold"><figcaption><p>A proposal: what we do, whether the test can see it, and what you will know</p></figcaption></figure>

Does a card say **Not measurable yet**? Then AdPage explains why and what has to happen first, usually moving more budget or choosing a bigger test group.
{% endstep %}

{% step %}

### Check the regions

Select **Show the design**. You see which provinces are in the test and which form the control group. **Experiment proposal** has the full proposal, as a PDF or by mail to your ads specialist.

<figure><img src="https://data.adpage.io/help/experimenten/03-ontwerp.png" alt="The test regions and the control regions of the design"><figcaption><p>Test regions and control regions</p></figcaption></figure>

Under **Try a different test group** you can pick other provinces and see roughly where the threshold lands. That is an estimate. Want to run the test with other provinces than the card shows? Ask AdPage for a new design before you start.
{% endstep %}

{% step %}

### Start the experiment

Select **Start this experiment** and confirm with **Show the runbook**. Nothing changes in your ad account.

<figure><img src="https://data.adpage.io/help/experimenten/04-starten.png" alt="The confirmation after Start this experiment"><figcaption><p>Starting changes nothing in your ad account</p></figcaption></figure>

Starting **records** the test regions, the control regions and the threshold of the card. The measurement always uses this record, also when AdPage recalculates the design later. Check that the provinces on the card are the provinces you set up in your ad account. As long as you have not done anything yet, **Move back to proposal** undoes the start.
{% endstep %}

{% step %}

### Set it up in your ad account

The runbook shows on a map where the test runs, how to set it up step by step, and what to check after 48 hours.

* **Holdout**: exclude the test provinces as a location in every campaign of the channel, and lower the budget so the control provinces get the same as before. Otherwise the budget that comes free flows to the control provinces, and the measured effect comes out too big.
* **Scale-up**: a separate campaign that only targets the test provinces, next to your national campaigns.
  {% endstep %}

{% step %}

### Fill in the logbook

At the bottom of the runbook is the **Logbook**. The top row is the day and time the change went live, the bottom row the day it was undone. These two dates are the measurement: without them there is nothing to measure.

<figure><img src="https://data.adpage.io/help/experimenten/06-logboek.png" alt="The logbook with the start date filled in"><figcaption><p>The start date is where the measurement begins</p></figcaption></figure>

Does AdPage see in the ad account on which day the test campaigns started delivering? Then it offers that date: select **Put it in the logbook**.
{% endstep %}

{% step %}

### Let it run

The card shows how far the test is and until when it runs. Leave the budgets, the regional targeting and the other channels as they are; new creatives and the daily optimisation can go on.

<figure><img src="https://data.adpage.io/help/experimenten/05-loopt.png" alt="A running experiment with its progress"><figcaption><p>A running test, with the date it runs until</p></figcaption></figure>

* Something regional happens, such as delivery problems or a local campaign? Select **Report a change**.
* Do not steer on numbers in between: the result is only reliable at the end. If the card shows moments to look, **Fetch the interim result** works on those moments only.
  {% endstep %}

{% step %}

### Fetch the result

Fill in the bottom row of the logbook, and select **Fetch the result** once the last week is over (from the Monday after). AdPage measures, shows the iROAS with its margin, and calibrates the model with it.

<figure><img src="https://data.adpage.io/help/experimenten/08-uitkomst.png" alt="The outcome of a measured experiment"><figcaption><p>A measured outcome, and what the model now sees</p></figcaption></figure>
{% endstep %}
{% endstepper %}

## Record a test that is already running

A test that started without its regions being recorded is not measured: the card says so. In the runbook, select **Record the current setup**, select the provinces that are in the test, and select **Record**. The other provinces become the control group. If AdPage recognises the split in your ad account, it suggests the provinces: select **Use this**.

<figure><img src="https://data.adpage.io/help/experimenten/07-vastleggen.png" alt="Recording the test provinces of a running test"><figcaption><p>Pick the provinces that are in the test, then Record</p></figcaption></figure>

This only works as long as nothing has been recorded for the test yet.

## Black Friday, Christmas and promotions

AdPage does not correct these weeks separately, and it does not need to: the control provinces go through the same weeks. With a year or more of history, last year's peak weeks are in the weeks the test learns from, so it knows how the test provinces normally relate to the rest in those weeks.

What does spoil the measurement is anything that does not happen in all provinces alike:

* **Keep promotions and discounts national.** Never run them in a few provinces only.
* **Keep the tested channel in the control provinces at its earlier level**, also around Black Friday. AdPage checks this afterwards: if the spend in the control provinces moves more than 15% from the weeks before, the outcome says so.
* **Raising other channels nationally is fine**, as long as every province gets it.
* **Report regional events** with **Report a change**.

## Troubleshooting

**"It has not been recorded which regions this test runs."** Select **Record the current setup** in the runbook.

**"The start date is not in the logbook yet."** Fill in the top row of the logbook, or take the date AdPage offers from the ad account.

**The card says Not measurable yet.** The test moves too little budget to see an effect. Move more budget, choose more provinces or run it longer, and ask AdPage for a new design.

**Starting says another test is still running.** Finish that test first: fetch its result.

**The provinces in your ad account differ from the card.** Do not start the test. Ask <support@adpage.io> for a design with your provinces.

Still stuck? Contact <support@adpage.io> and include the container and the name of the experiment.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.adpage.io/data-and-insights/data-studio/run-an-experiment.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
