We regularly meet customers who say that they are positively surprised about our methodology for incrementality measurement.

The most common awe we get are:

“Wait! Do you mean that I won't need to run incrementality experiments?”

“How do you find ground truth with so much noise in the data?”

“We won’t need to stop our campaigns?”

There are five ways to do incrementality testing: Randomized Control Group, Surveys & Panels, Blackouts, Partial Blackouts, and Causal Inference.

Four of these methods require experimentation - i.e. the customer does need to take an action in order to run the test.

With Randomized Control Group - a marketer needs to create an audience split and deliver PSA ads to a control group. This method did not survive in the privacy era.

Surveys & Panels - requires a significant investment in buying media to promote the survey, only to represent a fraction of the users and rely on users being able to explain why they (subconsciously) became customers.

Blackouts and Partial blackouts are painful to a marketer. And more importantly - they can often provide nonsense conclusions.

What are Planned Incrementality Experiments ?

The most common planned incrementality experiments will require a marketer to stop their advertising activities in a region, or market, for some time.

The purpose in this experiment is to find the marketing results ground truth.

Ground Truth is referred to as the number of users / sales that would happen if no Advertising activities are active.

The purpose in a ground truth test is to reach a point where sales numbers are not skewed, allowing to remove all noise from the data.

After reaching ground truth, a marketer would reactivate media vendor by vendor, one at a time to confidently calculate the incremental value of each media vendor.

We call BS.

Unless the ground truth test wipes the minds of all users in a market, or maybe if you’re running the experiment in the vacuum of space - marketing data is not deterministic, not linear and not as easy to measure as 0+1+2=3

Ground truth simply does not exist without a total reset.

Controlled experiments may give a marketer a sense of control, but measurement in the age of privacy requires a deeper consideration to factor in ad vendor audience overlap, seasonality factors, competition factors, and any external factor that could influence marketing performance.

While performing a blackout, organic results may be influenced due to various factors: your app getting featured, a new version of your app, competition investing heavily, or influencers such as weather, seasonality and other more obscure factors.

Attempting to run a true A/B using a blackout is simply an improbability.

Running Incrementality Tests Without Experiments

The solution we came up with relied on a few revelations: Marketers already make a lot of changes to their campaigns already. There’s no need to plan experiments, when marketers already make dozens of micro-experiments when optimizing campaigns, adjusting budgets, launching new creative sets, and activating new media vendors.

Our technology tracks every change a marketer makes to create millions of time series data, combining changes, to run retroactive predictions over what would happen if the change never happened? To provide clear and confident incrementality results.

Using time series, allowed our algorithm to account short and long term effects such as seasonality, holidays, day of the week and any external factor that has an influence over your marketing results.

Our methodology allowed marketers to use our platform and run continuous incrementality measurements without any need to run planned experiments.

Marketers could just click “run test” allowing our algorithm to calculate (in seconds!) the incrementality of a channel, campaign, ad group, or a single marketing change.

INCRMNTAL is a continuous incrementality measurement platform. Our software allows marketers to test out incrementality with a push of a button, and get results within seconds, with no need to run any experiments that require you to stop your campaigns. 

If you want to learn more, visit INCRMNTAL or book a demo today!