Google Research

Evaluating web search using task completion time

SIGIR '09: Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, ACM, New York, NY, USA (2009), pp. 676-677

Abstract

We consider experiments to measure the quality of a web search algorithm based on how much total time users take to complete assigned search tasks using that algorithm. We first analyze our data to verify that there is in fact a negative relationship between a user's total search time and a user's satisfaction for the types of tasks under consideration. Secondly, we fit a model with the user's total search time as the response to compare two different search algorithms. Finally, we propose an alternative experimental design which we demonstrate to be a substantial improvement over our current design in terms of variance reduction and efficiency.

Learn more about how we do research

We maintain a portfolio of research projects, providing individuals and teams the freedom to emphasize specific types of work