Courtney Heldreth
Research Areas
Authored Publications
Sort By
Believing Anthropomorphism: Examining the Role of Anthropomorphic Cues on User Trust in Large Language Models
Michelle Cohn
Femi Olanubi
Zion Mengesha
Daniel Padgett
CM (Association of Computing Machinery) CHI conference on Human Factors in Computing Systems 2024 (2024)
Preview abstract
People now regularly interface with Large Language Models (LLMs) via speech and text (e.g., Bard) interfaces. However, little is known about the relationship between how users anthropomorphize an LLM system (i.e., ascribe human-like characteristics to a system) and how they trust the information the system provides. Participants (n=2,165; ranging in age from 18-90 from the United States) completed an online experiment, where they interacted with a pseudo-LLM that varied in modality (text only, speech + text) and grammatical person (“I” vs. “the system”) in its responses. Results showed that the “speech + text” condition led to higher anthropomorphism of the system overall, as well as higher ratings of accuracy of the information the system provides. Additionally, the first-person pronoun (“I”) led to higher information accuracy and reduced risk ratings, but only in one context. We discuss these findings for their implications for the design of responsible, human–generative AI experiences.
View details
Preview abstract
There is increasing concern that how researchers currently define and measure fairness is inadequate. Recent calls push to move beyond traditional concepts of fairness and consider related constructs through qualitative and community-based approaches, particularly for underrepresented communities most at-risk for AI harm. One in context, previous research has identified that voice technologies are unfair due to racial and age disparities. This paper uses voice technologies as a case study to unpack how Black older adults value and envision fair and equitable AI systems. We conducted design workshops and interviews with 16 Black older adults, exploring how participants envisioned voice technologies that better understand cultural context and mitigate cultural dissonance. Our findings identify tensions between what it means to have fair, inclusive, and representative voice technologies. This research raises questions about how and whether researchers can model cultural representation with large language models.
View details
A Systematic Review and Thematic Analysis of Community-Collaborative Approaches to Computing Research
Ned Cooper
Tiffanie Horne
Gillian Hayes
Jess Scon Holbrook
Lauren Wilcox
ACM Conference on Human Factors in Computing Systems (ACM CHI) 2022 (2022)
Preview abstract
HCI researchers have been gradually shifting attention from individual users to communities when engaging in research, design, and system development. However, our field has yet to establish a cohesive, systematic understanding of the challenges, benefits, and commitments of community-collaborative approaches to research. We conducted a systematic review and thematic analysis of 47 computing research papers discussing participatory research with communities for the development of technological artifacts and systems, published over the last two decades. From this review, we identified seven themes associated with the evolution of a project: from establishing community partnerships to sustaining results. Our findings suggest several tensions characterize these projects, many of which relate to the power and position of researchers, and the computing research environment, relative to community partners. We discuss the implications of our findings and offer methodological proposals to guide HCI, and computing research more broadly, towards practices that center a community.
View details
“Mixture of amazement at the potential of this technology and concern about possible pitfalls”: Public sentiment towards AI in 15 countries
Patrick Gage Kelley
Christopher Moessner
Aaron M Sedley
Allison Woodruff
Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, 44 (2021), pp. 28-46
Preview abstract
Public opinion plays an important role in the development of technology, influencing product adoption, commercial development, research funding, career choices, and regulation. In this paper we present results of an in-depth survey of public opinion of artificial intelligence (AI) conducted with over 17,000 respondents spanning fifteen countries and six continents. Our analysis of open-ended responses regarding sentiment towards AI revealed four key themes (exciting, useful, worrying, and futuristic) which appear to varying degrees in different countries. These sentiments, and their relative prevalence, may inform how the public influences the development of AI.
View details
Exciting, Useful, Worrying, Futuristic: Public Perception of Artificial Intelligence in 8 Countries
Patrick Gage Kelley
Christopher Moessner
Aaron Sedley
Andreas Kramm
David T. Newman
Allison Woodruff
AIES '21: Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (2021), 627–637
Preview abstract
As the influence and use of artificial intelligence (AI) have grown and its transformative potential has become more apparent, many questions have been raised regarding the economic, political, social, and ethical implications of its use. Public opinion plays an important role in these discussions, influencing product adoption, commercial development, research funding, and regulation. In this paper we present results of an in-depth survey of public opinion of artificial intelligence conducted with 10,005 respondents spanning eight countries and six continents. We report widespread perception that AI will have significant impact on society, accompanied by strong support for the responsible development and use of AI, and also characterize the public’s sentiment towards AI with four key themes (exciting, useful, worrying, and futuristic) whose prevalence distinguishes response to AI in different countries.
View details
"I don't think these devices are very culturally sensitive." - The impact of errors on African Americans in Automated Speech Recognition
Zion Mengesha
Juliana Sublewski
Elyse Tuennerman
Frontiers in Artificial Intelligence, 26 (2021)
Preview abstract
Automated speech recognition (ASR) converts language into text and is used across a variety of applications to assist us in everyday life, from powering virtual assistants, natural language conversations, to enabling dictation services. While recent work suggests that there are racial disparities in the performance of ASR systems for speakers of African American Vernacular English, little is known about the psychological and experiential effects of these failures paper provides a detailed examination of the behavioral and psychological consequences of ASR voice errors and the difficulty African American users have with getting their intents recognized. The results demonstrate that ASR failures have a negative, detrimental impact on African American users. Specifically, African Americans feel othered when using technology powered by ASR—errors surface thoughts about identity, namely about race and geographic location—leaving them feeling that the technology was not made for them. As a result, African Americans accommodate their speech to have better success with the technology. We incorporate the insights and lessons learned from sociolinguistics in our suggestions for linguistically responsive ways to build more inclusive voice systems that consider African American users’ needs, attitudes, and speech patterns. Our findings suggest that the use of a diary study can enable researchers to best understand the experiences and needs of communities who are often misunderstood by ASR. We argue this methodological framework could enable researchers who are concerned with fairness in AI to better capture the needs of all speakers who are traditionally misheard by voice-activated, artificially intelligent (voice-AI) digital systems.
View details
Preview abstract
Artificial intelligence (AI) offers opportunities to solve complex problems facing smallholder farmers in the Global South. However, there is currently a dearth of research and resources available to organizations and policy-makers for building farmer-centered AI systems. As technologists, we believe it is our responsibility to draw from and contribute to research on farmers’ needs, practices, value systems, social worlds, and daily agricultural ecosystem realities. Drawing from our own fieldwork experience and scholarship, we propose concrete future directions for building AI solutions and tools that are meaningful to farmers and will significantly improve their lives. We also discuss tensions that may arise when incorporating AI into farming ecosystems. We hope that a closer look into these research areas will serve as a guide for technologists looking to leverage AI to help smallholder farmers in the Global South.
View details
Preview abstract
AI is powerful and has the potential to deliver many benefits to the Nigerian economy. As such, the government needs to play an important role in partnering with industry and the community to ensure its deployment is safe, fair, and produces positive outcomes. Given the early stage of AI development in Nigeria, we believe it’s important to make sure that policy makers have a clear and consistent understanding of the current state of AI in Nigeria-- the state of current laws and regulations as it applies to AI, current applications of AI, and the challenges AI presents on a policy level. We also present areas where the government, in collaboration with wider civil society and AI practitioners, can play a crucial role in advancing AI in Nigeria. We hope this paper can help in evolving the discussion to address policy ideas and implementation of AI in Nigeria.
View details
