Sarah D'Angelo
Research Areas
Authored Publications
Sort By
Preview abstract
The evolution of AI is a pivotal moment in history, but it’s not the first time we have experienced technological advances that have changed how humans work. By looking at the advances in automobiles, we are reminded of the importance of focusing on our developers' needs and goals.
View details
Preview abstract
AI-powered software development tooling is changing the way that developers interact with tools and write code. However, the ability for AI to truly transform software development depends on developers' level of trust in the tools. In this work, we take a mixed methods approach to measuring the factors that influence developers' trust in AI-powered code completion. We identified that familiarity with AI suggestions, quality of the suggestion, and level of expertise with the language all increased acceptance rate of AI-powered suggestions. While suggestion length and presence in a test file decreased acceptance rates. Based on these findings we propose recommendations for the design of AI-powered development tools to improve trust.
View details
Creativity, Generative AI, and Software Development: A Research Agenda
Victoria Jackson
Bogdan Vasilescu
Daniel Russo
Paul Ralph
Maliheh Izadi
Rafael Prikladnicki
Anielle Lisboa
Andre van der Hoek
Preview abstract
Creativity has always been considered a major differentiator to separate the good from the great, and we believe the importance of creativity to software development will only increase as GenAI becomes embedded in developer tool-chains and working practices. This paper uses the McLuhan tetrad alongside scenarios of how GenAI may disrupt software development more broadly, to identify potential impacts GenAI may have on creativity within software development. The impacts are discussed along with a future research agenda comprising of six connected themes that consider how individual capabilities, team capabilities, the product, unintended consequences, society, and human aspects can be affected.
View details
Measuring Developer Experience with a Longitudinal Survey
Jessica Lin
Jill Dicker
IEEE Software(2024)
Preview abstract
At Google, we’ve been running a quarterly large-scale survey with developers since 2018. In this article, we will discuss how we run EngSat, some of our key learnings over the past 6 years, and how we’ve evolved our approach to meet new needs and challenges.
View details
Preview abstract
Software developers write code nearly everyday, ranging from simple straightforward tasks to challenging and creative tasks. As we have seen across domains, AI/ML based assistants are on the rise in the field of computer science. We refer to them as code generation tools or AI/ML enhanced software developing tooling; and it is changing the way developers write code. As we think about how to design and measure the impact of intelligent writing assistants, the approaches used in software engineering and the considerations unique to writing code can provide a different and complementary perspective for the workshop. In this paper, we propose a focus on two themes: (1) measuring the impact of writing assistants and (2) how code writing assistants are changing the way engineers write code. In our discussion of these topics, we outline approaches used in software engineering, considerations unique to writing code, and how the disciplines of prose writing and code writing can learn from each other. We aim to contribute to the development of a taxonomy of writing assistants that includes possible methods of measurement and considers factors unique to the domain (e.g. prose or code).
View details
Using Logs Data to Identify When Software Engineers Experience Flow or Focused Work
Ben Holtz
ACM CHI Conference on Human Factors in Computing Systems(2023) (to appear)
Preview abstract
Beyond self-report data, we lack reliable and non-intrusive methods for identifying flow. However, taking a step back and acknowledging that flow occurs during periods of focus gives us the opportunity to make progress towards measuring flow by isolating focused work. Here, we take a mixed-methods approach to design a logs based metric that leverages machine learning and a comprehensive collection of logs data to identify periods of related actions (indicating focus), and validate this metric against self-reported time in focus or flow using diary data and quarterly survey data. Our results indicate that we can determine when software engineers at a large technology company experience focused work which includes instances of flow. This metric speaks to engineering work, but can be leveraged in other domains to non-disruptively measure when people experience focus. Future research can build upon this work to identify signals associated with other facets of flow.
View details
Preview abstract
Creativity is an essential part of software engineering. In this column, we describe our approach and insights into how software developers define creativity in their work. Our findings suggest that creativity in software engineering centers on the concepts of clever reuse over pure novelty.
View details
Preview abstract
In this installment of Developer Productivity for Humans, we present two lines of research emphasizing the human experience in measuring developer productivity: the experience of flow or focus and the experience of friction during development.
View details
Building and Sustaining Ethnically, Racially, and Gender Diverse Software Engineering Teams: A Study at Google
Ella Dagan
Anita Sarma
Alison Chang
Jill Dicker
Emerson Murphy-Hill
The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE)(2023) (to appear)
Preview abstract
Teams that build software are largely demographically homogeneous. Without diversity, homogeneous perspectives dominate how, why, and for whom software is designed. To understand how teams can successfully build and sustain diversity, we interviewed 11 engineers and 9 managers from some of the most gender and racially diverse teams at Google, a large software company. Qualitatively analyzing the interviews, we found shared approaches to recruiting, hiring, and promoting an inclusive environment, all of which create a positive feedback loop. Our findings produce actionable practices that every member of the team can take to increase diversity by fostering a more inclusive software engineering environment.
View details
Preview abstract
ML enhanced software development tooling is changing the way software engineers develop code. While the development of these tools continues to rise, studies have primarily focused on the accuracy and performance of underlying models, rather than the user experience. Understanding how engineers interact with ML enhanced tooling can help us define what successful interactions with ML based assistance look like. We therefore build upon prior research, by comparing software engineers' perceptions of two types of ML enhanced tools, (1) code completion and (2) code example suggestions. We then use our findings to inform design guidance for ML enhanced software development tooling. This research is intended to spark a growing conversation about the future of ML in software development and guide the design of developer tooling.
View details
