Estimating Ad Effectiveness using Geo Experiments in a Time-Based Regression Framework
Abstract
Two previously published papers (Vaver and Koehler, 2011, 2012) describe
a model for analyzing geo experiments. This model was designed to measure
advertising effectiveness using the rigor of a randomized experiment with replication
across geographic units providing confidence interval estimates. While effective, this
geo-based regression (GBR) approach is less applicable, or not applicable at all,
for situations in which few geographic units are available for testing (e.g. smaller
countries, or subregions of larger countries) These situations also include the so-called
matched market tests, which may compare the behavior of users in a single
control region with the behavior of users in a single test region. To fill this gap, we
have developed an analogous time-based regression (TBR) approach for analyzing
geo experiments. This methodology predicts the time series of the counterfactual
market response, allowing for direct estimation of the cumulative causal effect at
the end of the experiment. In this paper we describe this model and evaluate its
performance using simulation.
a model for analyzing geo experiments. This model was designed to measure
advertising effectiveness using the rigor of a randomized experiment with replication
across geographic units providing confidence interval estimates. While effective, this
geo-based regression (GBR) approach is less applicable, or not applicable at all,
for situations in which few geographic units are available for testing (e.g. smaller
countries, or subregions of larger countries) These situations also include the so-called
matched market tests, which may compare the behavior of users in a single
control region with the behavior of users in a single test region. To fill this gap, we
have developed an analogous time-based regression (TBR) approach for analyzing
geo experiments. This methodology predicts the time series of the counterfactual
market response, allowing for direct estimation of the cumulative causal effect at
the end of the experiment. In this paper we describe this model and evaluate its
performance using simulation.
Research Areas
