Martin Pielot
Martin Pielot is currently working as Quantitative User Experience Researcher (Quant UXR) at Google, ensuring that Google’s Material Design System helps designers and developers to create beautiful and usable user interfaces and experiences. In the past, he worked in the spaces of notifications & interruptions and tactile user interfaces. His publications received over 4000 citations and several best paper awards and honorable mentions.
Research Areas
Authored Publications
Google Publications
Other Publications
Sort By
Effects on reversing a 5 point vertically oriented satisfaction question, results from 3 studies using bipolar and unipolar satisfaction scales
European Survey Research Association 2023 Conference University of Milano Bicocca
Preview abstract
Customer satisfaction surveys are common in technology companies like Google. The standard satisfaction question asks respondents to rate how satisfied or dissatisfied they are with a product or service generally going from very satisfied to very dissatisfied.
When the scale is presented vertically, some survey literature suggests placing the positive end of the scale on top as “up means good” to avoid confusing respondents.
We report from 2 studies. The first study shows that reversing the response options of a bipolar satisfaction question (very dissatisfied on top) leads to significantly lower reported satisfaction. In a between group experiment, 3,000 Google Opinion Rewards (Smartphone panel) respondents took a 1-question satisfaction survey. When the response options were reversed participants were 10 times more likely to select the very dissatisfied option (5% versus 0.5% prevalence). They also took 11% more time to answer the reversed scale.
The second study shows that this effect can be partially explained by respondents mistaking the word dissatisfied for satisfied. ~1750 people responded to a reversed satisfaction question in an in-product survey on fonts.google.com. In a follow-up verification question (“You selected [answer option], was this your intention?”), 42.1% of the respondents indicated that they had selected very dissatisfied by mistake. Open ended feedback suggests that respondents hadn’t read properly and expected the positive option on top.
More experiments should be conducted on different samples to better understand the interaction of scale orientation versus the type of scale (unipolar vs. bipolar).
View details
No Results Found
