Incrementality testing: how to prove what your ads really add

Key takeaways

• Attribution counts credit; incrementality measures cause. They often disagree, sometimes by a lot.
• Geo holdouts are the workhorse: switch a channel off in matched regions and compare sales.
• Branded search is the most common place budget is wasted. Many of those clicks would have come for free.
• Open-source tools (Meta’s GeoLift, Google’s Meridian) put rigorous testing within reach of mid-sized advertisers.
• Rule: no budget increase for a channel until it beats a holdout.

The problem with platform numbers

Every ad platform reports conversions it was “involved” in. A customer who searched your brand name, clicked a paid ad and bought gets counted as a paid-search sale, even if they’d have clicked the free listing right below it. Add retargeting, view-through attribution and overlapping platforms, and the reported conversions often add up to more than your actual sales.

“…returns from paid search are a fraction of conventional non-experimental estimates.”

Blake, Nosko & Tadelis, NBER Working Paper 20171 (eBay field experiments)

Incrementality asks a different question: if we hadn’t run this, how many fewer sales would we have had?

Method 1: geo holdouts

Plan for 3 to 6 weeks per test, with at least 2 comparable regions per group, so that normal week-to-week swings don’t swamp the effect you are measuring.

1. Pick matched regions. Split your markets (cities, emirates, states, postcodes) into groups with similar history and seasonality.
2. Hold out. Turn the channel off, or down, in the test group for a fixed period, usually 3–6 weeks.
3. Compare. Measure the difference in total sales, from your CRM rather than the ad platform, between test and control, against what history predicted.
4. Calculate. Incremental sales ÷ spend = incremental return on ad spend (iROAS).

Tools: Meta’s open-source GeoLift helps choose matched markets and estimate the effect. Google’s open-source Meridian supports marketing mix modelling that you can calibrate with these test results.

Works best when you sell across several cities or regions. For a single-city business, use time-based tests instead: alternate weeks on and off, with care for seasonality.

Method 2: branded-search pause tests

Run the pause for 2 weeks in a normal trading period, and compare total revenue and organic brand clicks with the 2 weeks before.

The evidence base is strong: eBay’s large-scale field experiments found brand-keyword ads had no measurable short-term benefit, published as NBER Working Paper 20171.

Bidding on your own brand name protects you from competitors, but often buys clicks you’d get anyway.

• Pause brand campaigns in a holdout region, or for set periods.
• Watch total brand traffic and sales (paid plus organic), not just paid clicks.
• If total sales barely move, most of that spend wasn’t incremental. Keep a small defensive budget where competitors bid on your name, and move the rest.

Method 3: platform lift studies

Meta and Google offer conversion lift studies that randomly withhold ads from a control group of users. They’re useful, and free for eligible accounts. They’re also run by the platform being measured, so treat them as one input and confirm with your own geo tests where you can.

What this looked like in practice

In an in-house role at a Dubai curtains, blinds and interiors company, judging campaigns on booked measurements and orders, rather than on platform-reported conversions, was central to how budget moved. The account reached a 5.2× ROAS with acquisition cost down 30%. Those are previous-employer results. See the 30-day attribution audit.

Measuring when cookies are scarce

Click data is shrinking at the source: users clicked a traditional result on 8% of Google visits with an AI summary, against 15% without, according to Pew Research (2025).

Consent rules in Europe and browser restrictions mean platforms see fewer conversions directly. Geo-based tests don’t rely on tracking individuals, which makes them more robust in a privacy-first world. See consent-based measurement in Europe.