Incrementality measurement is the gold standard of Advertising measurement. Measuring value, rather than measuring traffic.
Incrementality measurement importance grew rapidly due to the infestation of attribution fraud over the past several years. With the deprecation of identifiers (IDFA, Cookies, ITP, GAID) Incrementality measurement is most accurate way of measuring and optimizing marketing spend.

There are various ways Advertisers use to measure incrementality.
We wanted to provide an objective view over the various methods:

A/B Testing:

The process of A>B testing is a relatively simple one - creating two campaigns or line items, where one would show users PSA or “ghost ads” while the second campaign displays the normal creatives.
The purpose of this test is to create an uncontrolled split in the audience and compare the performance of the two campaigns.

Pro’s:

- Easy to execute and understand the results of the test
- Even if the results are not very accurate - they do provide an indication

Cons:

- Relevant for Advertisers working only with 1 vendor, as spillage would influence the result of the test
- The advertiser must waste media costs over PSA or Ghost ads
- Requires the ability to create user level frequency capping
- Not a valid option in a post-IDFA world

GEO Split
 

A GEO split is extremely common for FMCG multi-country brands, as market penetration cost are exceptionally high and require an upfront investment.

The method requires the Advertiser to control a product promotion or launch while making the product available, but limiting the promotion and access to the product in another GEO.
This method is commonly used in Media Mix Modeling
 

Pro’s:

- Suitable for Brands where market penetration requires a large upfront investment
- Easy to execute the test for mobile and digital only advertisers
 

Con’s:

- Markets will behave differently due to various factors that may not be examined leading to wrong assumptions
- The results of the test will be far from conclusive for operational / tactical decision making.

Randomized Control Group

Very similar to A>B Incrementality testing and GEO Split - a randomized control group seeks to split the user base being advertised to in a homogeneous way, so that there is an almost equilibrium between group A and group B.

The groups are served with either PSA ads or the campaign creatives and the performance is measured to reach a conclusive outcome.
 

Pro’s:

- This method can provide rather conclusive results as long as the Advertiser can fully control the targeting limiting against any spillage or overlap
 

Con’s:

- Given the requirement to identify “users” on publishers websites and apps - this method became obsolete as the % of LAT users (Limit Ad Tracking) grew, as well as Apple deprecating IDFA
- Performing this test in a single platform rather than across all media platforms used will not provide a reliable outcome.

Switching off Advertising

Probably the easiest and most effective way to measure incrementality in advertising is to stop advertising.

It may sound ridiculous, but Advertisers who experiment with this are able to improve their advertising effectiveness.

The method requires Advertisers to stop ALL advertising activities for some time, to allow a true “baseline” to be identified, followed by a slow, gradual process, of reactivating campaigns and media vendors, while monitoring the effectiveness of each one in an isolated manner.

Pro’s:

- The most effective method to reach a conclusive outcome
- No additional media costs
 

Con’s:

- Opportunity cost may outweigh the outcome
- Any external factors influencing the results may cause the test to be flawed
- The test needs to be performed multiple times in order to continuously measure effectiveness

Causal Inference

The most recent method for incrementality measurement. Causal Inference is an algorithmic process to draw conclusions about the causality of results in a multi-variant and noisy environment.
Causal inference is used in epidemiological research, in economics and most recently, in AI.
Applying causal inference models requires research and development to come up with hyper-parameters across a time series to provide a digestible insight over the value of marketing activities.

Pro’s:

- Causal inference can digest digital as well as offline campaign activities
- Actionable insights can be granular enough to provide operational suggestions
- Campaign data is used as is without the need for any disruption
- Resilient to a post-IDFA world
 

Con’s:

- Requires significant data research to reach the right hyper-parameters
- A one of research would become incorrect over time (continuous research needed)


We hope that this guide was useful for you.

You may find more information about incrementality in these resources:
Causal inference in statistics: An overview
Incrementality bidding & attribution
Semiparametric Difference in Difference Estimators
A Unique method for ML interpretability: Game Theory & Shapley Value


INCRMNTAL is an incrementality measurement platform. Our platform uses machine learning and causality algorithms to provide you with incrementality and cannibalization actionable insights.
Our platform provides insights over all paid marketing activities, even the untrackable ones.

If you want to learn more, visit INCRMNTAL or book a demo today!