Your test
Every region needs a value on every date (use 0 when nothing sold). Use one period length throughout, daily or weekly. Regions not named as test regions are the comparison group and should have had no campaign. At least 14 periods before the start, 3 after, and one comparison region are needed.
Upload or paste your table, name the test regions and the start date, then click Measure lift.
Planning a test? Pick the regions before you spend
Paste (or load) sales history by region in the box above, with no campaign running in it. The planner tries different splits of regions into campaign and comparison groups, replays fake campaigns on your history for each, and ranks the splits by the smallest lift the test could detect. Choose the split that can see a lift of the size you expect.
Needs at least 28 periods of history and three regions.
| Rank | Run the campaign in | Detectable lift | Fit (R²) | Share of sales |
|---|
Detectable lift by campaign length (best split)
Test plan: how long must the campaign run?
Uses only your history, as if no campaign had run. Name the test regions above (no start date needed). For each campaign length it shows the smallest lift the test could tell from noise.
| Campaign length (periods) | Smallest detectable lift | Placebo windows |
|---|
Result
Actual against what would have happened
What the money says
Second opinion and comparison weights
| Comparison region | Weight |
|---|
How it is calculated
Counterfactual: before the start date, total test-region value is regressed on the comparison regions with non-negative weights (no intercept, weights free in scale), by coordinate-descent least squares. After the start date the same weights predict the no-campaign value. Lift is actual minus predicted, added over the campaign and divided by the predicted total for the relative lift. Interval: for every possible campaign-length window in the pre-period the model is refitted on earlier days only (or on the days either side when history is short) and the fake cumulative gap is recorded. The 90% interval is the real lift minus the 5th and 95th percentiles of those fake gaps; the p-value is the share of fake gaps at least as large as the real one. Overlapping windows are not independent, so treat both as approximate. Second opinion: the test total compared with the pooled comparison total, scaled by their pre-campaign ratio (difference in differences).
Not modelled: spillover between regions (ads reaching people in a comparison region), promotions or stock-outs that hit only one side, price changes, long-run brand effects after the test, and effects that ramp up slowly. Regions chosen because they were doing well are not a fair test. Unverified: checked against simulated panels with known lifts in unit tests, not against a published marketing experiment.
Common questions
What is an ad incrementality test?
It asks whether your ads caused sales that would not have happened anyway. You switch a campaign on in some stores or regions and leave comparable ones alone, then compare what happened. The difference, beyond what the comparison regions predicted, is the incremental effect. Platform-reported conversions count everyone who saw an ad and bought, including people who were going to buy regardless.
How do I run a geo holdout test for my ads?
Pick several regions, cities or stores with steady sales, run the campaign in some of them and keep others dark for at least two to four weeks. Collect daily or weekly sales for all of them from well before the start date. Paste that table here with the start date and the names of the test regions.
How is the counterfactual built?
Sales in the test regions are explained before the campaign as a non-negative weighted sum of the comparison regions, fitted by least squares. The same weights are then used after the start date to predict what the test regions would have sold with no campaign. The gap between actual and predicted is the lift. A simple before-and-after comparison against the pooled comparison regions is shown as a second opinion.
How sure is the result?
The tool replays the method on the days before your campaign: it pretends the campaign started earlier, predicts forward, and records how big a fake lift can look from noise alone. Your real lift is compared with those fake ones, which gives an interval and a p-value. A wide interval means the test was too small or too noisy to say; the tool shows the smallest lift it could have detected.
How many regions and days do I need?
At least 14 days before the campaign, 3 or more after it, and at least one comparison region. In practice four or more comparison regions that move like the test ones, several weeks of history before the start and a campaign that runs two to four weeks give a usable answer. If the comparison regions do not track the test regions beforehand the tool says the result is unreliable.
What is iROAS and why is it lower than my platform ROAS?
Incremental return on ad spend is extra sales caused by the ads divided by ad spend. Platforms credit every conversion that follows an ad touch, including buyers who would have bought anyway, so their ROAS is usually higher. Break-even iROAS is one divided by your gross margin: at a 40% margin you need 2.5 dollars of extra sales per dollar of spend to break even.
How do I choose which regions to run my ad test in?
Pick the split whose comparison regions best predict the test regions, so that a small real lift stands out from noise. The planner on this page replays fake campaigns on your own sales history for many possible splits and ranks them by the smallest lift each could detect. Avoid test regions that swing on their own (a tourist town, one with frequent local promotions) and prefer a split where the test regions are not so large a share of sales that the rest cannot predict them.
Is my sales data uploaded anywhere?
No. The file is read and analysed in this browser tab. Nothing is sent to a server, there are no accounts or cookies, and the tool never stores your data. It works offline once the page has loaded.