Mobile Survey Scale Showdown: Stars vs Smileys

With increased adoption and usage of mobile apps for a variety of purposes, it is important to establish attitudinal measurement designs to measure users’ experiences in context of actual app usage. Such designs should balance mobile UX considerations with survey data quality. To inform choices on contextual mobile survey design, we conduct a comparative evaluation of stars vs smileys as graphical scales for in-context mobile app satisfaction measurement, as follows: To evaluate and compare data quality across scale types, we look at the distributions of the numerical ratings by anchor point stimulus to evaluate the extremity and scale point distances. We also assess criterion validity for stars and smileys, where feasible. To evaluate User Experience across variants, we compare key survey-related signals such as response & dismiss rates, dismiss/response ratio, and time-to-response.

• Algorithms and Theory

• Human-Computer Interaction and Visualization