Google Research

StrategicReading: Understanding Complex Mobile Reading Strategies via Implicit Behavior Sensing

Proceedings of ACM International Conference on Multimodal Interaction (ICMI 2020), ACM


Mobile devices are becoming an important platform for reading. However, existing research on mobile reading primarily focuses on low-level metrics such as speed and comprehension. In particular, for complex reading tasks involving information seeking, source evaluation, and integrative comprehension, researchers still rely on the labor-intensive analysis of reader-generated verbal reports. We present StrategicReading, an intelligent reading system running on unmodified smartphones, to understand high-level strategic reading behaviors on mobile devices. StrategicReading leverages multimodal behavior sensing and takes advantage of signals from camera-based gaze sensing, kinematic scrolling patterns, and the evolution of cross-page behaviors. Through a 40-participant study, we found that gaze patterns, muscle stiffness signals, and reading paths captured by StrategicReading can infer both users' reading strategies and reading performance with high accuracy.

Learn more about how we do research

We maintain a portfolio of research projects, providing individuals and teams the freedom to emphasize specific types of work