Joe Moran

I am a UX Researcher in the CoreML pillar at Google, where I work on building the tools that drive ML excellence for Google and the world. I am particularly interested in understanding how production ML development works: the workflows, use cases, and experimentation loops that engineers engage with to bring their ML products to life.
Authored Publications
Sort By
  • Title
  • Title, descending
  • Year
  • Year, descending
Preview abstract This paper examines how English-speaking adults in the United States (n=1530) anthropomorphize and calibrate trust for a pseudo-large language model (LLM). We compared four generations: Gen Z (age 18-26), Millennials (age 27-43), Gen X (age 44-57), and Baby Boomers (age 58-78). The LLM varied in cues of anthropomorphism: whether the system responded in text-only or text and speech, as well as whether it used the first-person singular (“I”) or third-person singular (“The system”) in its responses. Across both types of manipulations, results showed that Gen Z, in particular, showed consistently lower anthropomorphism (less competent, human-like, natural, conscious, life-like), reduced trust ratings of the system, and lower ratings of accuracy for responses generated by the LLM. Furthermore, we saw that the likelihood to externally validate the LLM’s response varied across the generations across contexts, where Baby Boomers were more likely to validate information about medication and health disorders and Gen Z about cooking. We did not see an interaction between the generations and the anthropomorphic cues manipulated; rather, there was a consistent increase in anthropomorphism and perceived accuracy for systems that used speech + text, relative to text only. There were no across-the-board effects of the first-person singular. We discuss these findings in terms of their implications for responsible AI. View details
Believing Anthropomorphism: Examining the Role of Anthropomorphic Cues on User Trust in Large Language Models
Michelle Cohn
Femi Olanubi
Zion Mengesha
Daniel Padgett
CM (Association of Computing Machinery) CHI conference on Human Factors in Computing Systems 2024 (2024)
Preview abstract People now regularly interface with Large Language Models (LLMs) via speech and text (e.g., Bard) interfaces. However, little is known about the relationship between how users anthropomorphize an LLM system (i.e., ascribe human-like characteristics to a system) and how they trust the information the system provides. Participants (n=2,165; ranging in age from 18-90 from the United States) completed an online experiment, where they interacted with a pseudo-LLM that varied in modality (text only, speech + text) and grammatical person (“I” vs. “the system”) in its responses. Results showed that the “speech + text” condition led to higher anthropomorphism of the system overall, as well as higher ratings of accuracy of the information the system provides. Additionally, the first-person pronoun (“I”) led to higher information accuracy and reduced risk ratings, but only in one context. We discuss these findings for their implications for the design of responsible, human–generative AI experiences. View details
×