Take Action

Home | Faculty & Research Overview | Research

Research Details

Predictive Incrementality by Experimentation (PIE) for Ad Measurement

Abstract

We present a novel approach to causal measurement for advertising, namely to use exogenous variation in advertising exposure (RCTs) for a subset of ad campaigns to build a model that can predict the causal effect of ad campaigns that were run without RCTs. This approach -- Predictive Incrementality by Experimentation (PIE) -- frames the task of estimating the causal effect of an ad campaign as a prediction problem, with the unit of observation being an RCT itself. In contrast, traditional causal inference approaches with observational data seek to adjust covariate imbalance at the user level. A key insight is to use post-campaign features, such as last-click conversion counts, that do not require an RCT, as features in our predictive model. We find that our PIE model recovers RCT-derived incremental conversions per dollar (ICPD) much better than the program evaluation approaches analyzed in Gordon et al. (forthcoming). The prediction errors from the best PIE model are 48%, 42%, and 62% of the RCT-based average ICPD for upper-, mid-, and lower-funnel conversion outcomes, respectively. In contrast, across the same data, the average prediction error of stratified propensity score matching exceeds 491%, and that of double/debiased machine learning exceeds 2,904%. Using a decision-making framework inspired by industry, we show that PIE leads to different decisions compared to RCTs for only 6% of upper-funnel, 7% of mid-funnel, and 13% of lower-funnel outcomes. We conclude that PIE could enable advertising platforms to scale causal ad measurement by extrapolating from a limited number of RCTs to a large set of non-experimental ad campaigns.

Type

Working Paper

Author(s)

Brett Gordon, Robert Moakler, Florian Zettelmeyer

Date Published

2024

Citations

Gordon, Brett, Robert Moakler, and Florian Zettelmeyer. 2024. Predictive Incrementality by Experimentation (PIE) for Ad Measurement.

LINK
KELLOGG INSIGHT

Explore leading research and ideas

Find articles, podcast episodes, and videos that spark ideas in lifelong learners, and inspire those looking to advance in their careers.
learn more

COURSE CATALOG

Review Courses & Schedules

Access information about specific courses and their schedules by viewing the interactive course scheduler tool.
LEARN MORE

DEGREE PROGRAMS

Discover the path to your goals

Whether you choose our Full-Time, Part-Time or Executive MBA program, you’ll enjoy the same unparalleled education, exceptional faculty and distinctive culture.
learn more