Periodic Measurement of Advertising Effectiveness Using Multiple-Test-Period Geo Experiments

Jim Koehler
Google Inc. (2012)
Google Scholar

Abstract

In a previous paper [6] we described the application of geo experiments to the measurement of advertising effectiveness. One reason this method of measurement is attractive is that it provides the rigor of a randomized experiment. However, related decisions, such as where and how to spend advertising budget, are not static. To address this issue, we extend this methodology to provide periodic (ongoing) measurement of ad effectiveness. In this approach, the test and control assignments of each geographic region rotate across multiple test periods, and these rotations provide the opportunity to generate a sequence of measurements of campaign effectiveness. The data across test periods can also be pooled to create a single aggregate measurement of campaign effectiveness. These sequential and pooled measurements have smaller confidence intervals than measurements from a series of geo experiments with a single test period. Alternatively, the same confidence interval can be achieved with a reduced magnitude and/or duration of ad spend change, thereby lowering the cost of measurement. The net result is a better method for periodic and isolated measurement of ad effectiveness.