Jump to Content
Caitlin Sadowski

Caitlin Sadowski

Dr. Caitlin Sadowski is an engineering manager at Google in Mountain View, California, where she leads the Chrome Data team: a cross-functional team focused on tools, infrastructure, and insights related to measuring Chrome browser usage. In the past, she made static analysis useful at Google by creating the Tricorder program analysis platform and co-founded a team focused on understanding developer productivity. She has a PhD from the University of California at Santa Cruz where she worked on a variety of research topics related to programming languages, software engineering, and human computer interaction. She enjoys baking with her two young children.
Authored Publications
Google Publications
Other Publications
Sort By
  • Title
  • Title, descending
  • Year
  • Year, descending
    A World Wide View of Browsing the World Wide Web
    Aurore Fass
    Emma Thomas
    Jon Azose
    Kimberly Ruth
    Mark Pearson
    Zakir Durumeric
    Internet Measurement Conference (IMC), ACM (2022) (to appear)
    Preview abstract In this paper, we perform the first large-scale study of how people use and spend time on the web. Our study is based on anonymous, aggregated telemetry data from a major web browser with several hundred million users globally. Our study considers only users who chose to explicitly enable sharing URLs with the browser vendor and have usage statistic reporting enabled. We analyze the distribution of web traffic, the types of popular sites that people visit and spend the most time on, differences between desktop and mobile browsing behavior, geographical differences in web usage, and the websites popular in regions worldwide. Our study sheds light on both online user behavior and how the measurement community can better analyze the web in future research studies. View details
    Enabling the Study of Software Development Behavior with Cross-Tool Logs
    Ben Holtz
    Edward K. Smith
    Andrea Marie Knight Dolan
    Elizabeth Kammer
    Jillian Dicker
    Lan Cheng
    IEEE Software, vol. Special Issue on Behavioral Science of Software Engineering (2020)
    Preview abstract Understanding developers’ day-to-day behavior can help answer important research questions, but capturing that behavior at scale can be challenging, particularly when developers use many tools in concert to accomplish their tasks. In this paper, we describe our experience creating a system that integrates log data from dozens of development tools at Google, including tools that developers use to email, schedule meetings, ask and answer technical questions, find code, build and test, and review code. The contribution of this article is a technical description of the system, a validation of it, and a demonstration of its usefulness. View details
    Do Developers Learn New Tools On The Toilet?
    Edward K. Smith
    Andrea Knight Dolan
    Andrew Trenk
    Steve Gross
    Proceedings of the 2019 International Conference on Software Engineering
    Preview abstract Maintaining awareness of useful tools is a substantial challenge for developers. Physical newsletters are a simple technique to inform developers about tools. In this paper, we evaluate such a technique, called Testing on the Toilet, by performing a mixed-methods case study. We first quantitatively evaluate how effective this technique is by applying statistical causal inference over six years of data about tools used by thousands of developers. We then qualitatively contextualize these results by interviewing and surveying 382 developers, from authors to editors to readers. We found that the technique was generally effective at increasing software development tool use, although the increase varied depending on factors such as the breadth of applicability of the tool, the extent to which the tool has reached saturation, and the memorability of the tool name. View details
    What Predicts Software Developers’ Productivity?
    David C. Shepherd
    Michael Phillips
    Andrea Knight Dolan
    Edward K. Smith
    Transactions on Software Engineering (2019)
    Preview abstract Organizations have a variety of options to help their software developers become their most productive selves, from modifying office layouts, to investing in better tools, to cleaning up the source code. But which options will have the biggest impact? Drawing from the literature in software engineering and industrial/organizational psychology to identify factors that correlate with productivity, we designed a survey that asked 622 developers across 3 companies about these productivity factors and about self-rated productivity. Our results suggest that the factors that most strongly correlate with self-rated productivity were non-technical factors, such as job enthusiasm, peer support for new ideas, and receiving useful feedback about job performance. Compared to other knowledge workers, our results also suggest that software developers’ self-rated productivity is more strongly related to task variety and ability to work remotely. View details
    Web Feature Deprecation: A Case Study for Chrome
    Ariana Mirian
    Geoffrey M. Voelker
    Nik Bhagat
    Stefan Savage
    International Conference on Software Engineering (ICSE) SEIP track (2019) (to appear)
    Preview abstract Deprecation is a necessary function for the health and innovation of the web ecosystem. However, web feature deprecation is an understudied topic. While Chrome has a protocol for web feature deprecation, much of this process is based on a mix of few metrics and intuition. In this paper, we analyze web feature deprecations, in an attempt to improve this process. First, we produce a taxonomy of reasons why developers want to deprecate web features. We then provide a set of guidelines for deciding when it is safe to deprecate a web feature and a methodology for approaching the question of whether to deprecate a web feature. Finally, we provide a tool that helps determine whether a web feature meets these guidelines for deprecation. We also discuss the challenges faced during this process. View details
    When Not to Comment: Questions and Tradeoffs with API Documentation for C++ Projects
    Andrew Head
    Emerson Murphy-Hill
    Andrea Knight
    International Conference on Software Engineering (ICSE) (2018) (to appear)
    Preview abstract Without usable and accurate documentation of how to use an API, programmers can find themselves deterred from reusing relevant code. In C++, one place developers can find documentation is in a header file, but when information is missing, they may look at the corresponding implementation (the “.cc” file). To understand what’s missing from C++ API documentation and whether it should be fixed, we conducted a mixed-methods study. This involved three experience sampling studies with hundreds of developers at the moment they visited implementation code, interviews with 18 of those developers, and interviews with 8 API maintainers. We found that in many cases, updating documentation may provide only limited value for developers, while requiring effort maintainers don’t want to invest. This helps frame future tools and processes designed to fill in missing low-level API documentation. View details
    Preview abstract Software developers’ productivity can be negatively impacted by using APIs incorrectly. In this paper, we describe an analysis technique we designed to find API usability problems by comparing successive file-level changes made by individual software developers. We applied our tool, StopMotion, to the file histories of real developers doing real tasks at Google. The results reveal several API usability challenges including simple typos, conceptual API misalignments, and conflation of similar APIs. View details
    Modern Code Review: A Case Study at Google
    Emma Söderberg
    Luke Church
    Michal Sipko
    Alberto Bacchelli
    International Conference on Software Engineering, Software Engineering in Practice track (ICSE SEIP) (2018) (to appear)
    Preview abstract Employing lightweight, tool-based code review of code changes (aka modern code review) has become the norm for a wide variety of open-source and industrial systems. In this paper, we make an exploratory investigation of modern code review at Google. Google introduced code review early on and evolved it over the years; our study sheds light on why Google introduced this practice and analyzes its current status, after the process has been refined through decades of code changes and millions of code reviews. By means of 12 interviews, a survey with 44 respondents, and the analysis of review logs for 9 million reviewed changes, we investigate motivations behind code review at Google, current practices, and developers’ satisfaction and challenges. View details
    Lessons from Building Static Analysis Tools at Google
    Edward Aftandilian
    Alex Eagle
    Liam Miller-Cushon
    Communications of the ACM (CACM), vol. 61 Issue 4 (2018), pp. 58-66
    Preview abstract In this article, we describe how we have applied the lessons from Google’s previous experience with FindBugs Java analysis, as well as lessons from the academic literature, to build a successful static analysis infrastructure that is used daily by the majority of engineers at Google. Our tooling detects thousands of issues per day that are fixed by engineers, by their own choice, before the problematic code is checked into the codebase. View details
    Advantages and Disadvantages of a Monolithic Codebase
    Andrea Knight
    Edward K. Smith
    International Conference on Software Engineering, Software Engineering in Practice track (ICSE SEIP) (2018)
    Preview abstract Monolithic source code repositories (repos) are used by several large tech companies, but little is known about their advantages or disadvantages compared to multiple per-project repos. This paper investigates the relative tradeoffs by utilizing a mixed-methods approach. Our primary contribution is a survey of engineers who have experience with both monolithic repos and multiple, per-project repos. This paper also backs up the claims made by these engineers with a large-scale analysis of developer tool logs. Our study finds that the visibility of the codebase is a significant advantage of a monolithic repo: it enables engineers to discover APIs to reuse, find examples for using an API, and automatically have dependent code updated as an API migrates to a new version. Engineers also appreciate the centralization of dependency management in the repo. In contrast, multiple-repository (multi-repo) systems afford engineers more flexibility to select their own toolchains and provide significant access control and stability benefits. In both cases, the related tooling is also a significant factor; engineers favor particular tools and are drawn to repo management systems that support their desired toolchain. View details
    A Cross-Tool Study on Program Analysis Tool Notification Communication
    Brittany Johnson
    Rahul Pandita
    Justin Smith
    Denae Ford
    Sarah Elder
    Emerson Murphy-Hill
    Sarah Heckman
    Foundations of Software Engineering (FSE) (2016)
    Preview abstract Program analysis tools use notifications to communicate with developers, but previous research suggests that developers encounter challenges that impede this communication. This paper describes a qualitative study that identifies 12 kinds of challenges that cause notifications to miscommunicate with developers. Our resulting theory reveals that many challenges span multiple tools and multiple levels of developer experience. Our results suggest that, for example, future tools that model developer experience could improve communication and help developers build more accurate mental models. View details
    An Empirical Study of Practitioners’ Perspectives on Green Software Engineering
    Irene Manotas
    Christian Bird
    Rui Zhang
    David Shepherd
    Lori Pollock
    James Clause
    International Conference on Software Engineering (ICSE), 1600 Amphitheatre Parkway (2016) (to appear)
    Preview abstract The energy consumption of software is an increasing concern as the use of mobile applications, embedded systems, and data center-based services expands. While research in green software engineering is correspondingly increasing, little is known about the current practices and perspectives of software engineers in the field. This paper describes the first empirical study of how practitioners think about energy when they write requirements, design, construct, test, and maintain their software. We report findings from a quantitative, targeted survey of 464 practitioners from ABB, Google, IBM, and Microsoft, which was motivated by and supported with qualitative data from 18 in-depth interviews with Microsoft employees. The major findings and implications from the collected data contextualize existing green software engineering research and suggest directions for researchers aiming to develop strategies and tools to help practitioners improve the energy usage of their applications. View details
    How Developers Search for Code: A Case Study
    Kathryn T. Stolee
    Sebastian Elbaum
    Joint Meeting of the European Software Engineering Conference and the Symposium on the Foundations of Software Engineering (ESEC/FSE ), 1600 Amphitheatre Parkway (2015)
    Preview abstract With the advent of large code repositories and sophisticated search capabilities, code search is increasingly becoming a key software development activity. In this work we shed some light into how developers search for code through a case study performed at Google, using a combination of survey and log-analysis methodologies. Our study provides insights into what developers are doing and trying to learn when performing a search, search scope, query properties, and what a search session under different contexts usually entails. Our results indicate that programmers search for code very frequently, conducting an average of five search sessions with 12 total queries each workday. The search queries are often targeted at a particular code location and programmers are typically looking for code with which they are somewhat familiar. Further, programmers are generally seeking answers to questions about how to use an API, what code does, why something is failing, or where code is located. View details
    Preview abstract Static analysis tools help developers find bugs, improve code readability, and ensure consistent style across a project. However, these tools can be difficult to smoothly integrate with each other and into the developer workflow, particularly when scaling to large codebases. We present Tricorder, a program analysis platform aimed at building a data-driven ecosystem around program analysis. We present a set of guiding principles for our program analysis tools and a scalable architecture for an analysis platform implementing these principles. We include an empirical, in-situ evaluation of the tool as it is used by developers across Google that shows the usefulness and impact of the platform. View details
    RLint: Reformatting R Code to Follow the Google Style Guide
    Alex Blocker
    Andy Chen
    Andy Chu
    Tim Hesterberg
    Tom Zhang
    R User Conference (2014)
    Preview abstract RLint (https://code.google.com/p/google-rlint/) both checks and reformats R code to the Google R Style Guide. It warns of violations and optionally produces compliant code. It considers proper spacing, line alignment inside brackets, and other style violations, but like all lint programs does not try to handle all syntax issues. Code that follows a uniform style eases maintenance, modification, and ensuring correctness, especially when multiple programmers are involved. Thus, RLint is automatically used within Google as part of the peer review process for R code. We encourage CRAN package authors and other R programmers to use this tool. A user can run the open-source Python-based program in a Linux, Unix, Mac or Windows machine via a command line. View details
    Programmers’ Build Errors: A Case Study (at Google)
    Hyunmin Seo
    Sebastian Elbaum
    Edward Aftandilian
    Robert Bowdidge
    International Conference on Software Engineering (ICSE) (2014)
    Preview abstract Building is an integral part of the software development process. However, little is known about the errors occurring in this process. In this paper, we present an empirical study of 26.6 million builds produced during a period of nine months by thousands of developers. We describe the workflow through which those builds are generated, and we analyze failure frequency, error types, and resolution efforts to fix those errors. The results provide insights on how a large organization build process works, and pinpoints errors for which further developer support would be most effective. View details
    How Developers Use Data Race Detection Tools
    Evaluation and Usability of Programming Languages and Tools (PLATEAU), ACM (2014)
    Preview abstract Developers need help with multithreaded programming. We investigate how two program analysis tools are used by developers at Google: ThreadSafety, an annotation-based static data race analysis, and TSan, a dynamic data race detector. The data was collected by interviewing seven veteran industry developers at Google, and provides unique insight into how four different teams use tooling in different ways to help with multithreaded programming. The result is a collection of perceived pros and cons of using ThreadSafety and TSan, as well as general issues with multithreading. View details
    2nd international workshop on user evaluations for software engineering researchers (USER)
    Andrew Begel
    International Conference on Software Engineering (ICSE) (2013)
    Preview abstract We have met many software engineering researchers who would like to evaluate a tool or system they developed with real users, but do not know how to begin. In this second iteration of the USER workshop, attendees will collaboratively design, develop, and pilot plans for conducting user evaluations of their own tools and/or software engineering research projects. Attendees will gain practical experience with various user evaluation methods through scaffolded group exercises, panel discussions, and mentoring by a panel of user-focused software engineering researchers. Together, we will establish a community of likeminded researchers and developers to help one another improve our research and practice through user evaluation. View details
    Does Bug Prediction Support Human Developers? Findings from a Google Case Study
    Chris Lewis
    Zhongpeng Lin
    Xiaoyan Zhu
    Rong Ou
    E. James Whitehead Jr.
    International Conference on Software Engineering (ICSE) (2013)
    Preview abstract While many bug prediction algorithms have been developed by academia, they're often only tested and verified in the lab using automated means. We do not have a strong idea about whether such algorithms are useful to guide human developers. We deployed a bug prediction algorithm across Google, and found no identifiable change in developer behavior. Using our experience, we provide several characteristics that bug prediction algorithms need to meet in order to be accepted by human developers and truly change how developers evaluate their code. View details
    Proceedings of the ACM 4th annual workshop on Evaluation and usability of programming languages and tools (PLATEAU)
    Emerson Murphy-Hill
    Shane Markstrum
    Conference on Systems, Programming, and Applications: Software for Humanity (SPLASH) (2012)
    No Results Found