Smart Paste for context-aware adjustments to pasted code

Most developers copy and paste code frequently during their daily work to avoid unnecessary repetitive transcriptions. While this speeds up the process of code development, there is more to it than simply copy-pasting. An analysis of the Google monorepo (Google’s code repository, which stores the complete working history, not just the committed changes) reveals interesting patterns in user behaviors that are potential targets for efficiency improvements. For example, according to the edit history, ~25% of pastes are immediately modified, with edits ranging from small syntactic fixes (e.g., adding a missing semicolon) to more sophisticated adaptations to the surrounding code context (e.g., renaming a variable and changing its type). Making such changes often breaks the code-authoring flow and slows down the development process.

With this in mind, we present Smart Paste, a tool now broadly available at Google, that predicts the next state of a code environment and uses generative AI to create context-aware adjustments to pasted code. Leveraging recent advances in large sequence models — e.g., DIDACT, which has made possible tools for code review, build repair, and more — Smart Paste streamlines the process of copy-pasting during code revisions. In a usage study spanning ~40k engineers who adopted the feature, we found that 6.9% of all pastes in the IDE utilized Smart Paste, with a 42.5% acceptance rate, which significantly streamlined user workflow.

For model training, we sourced data of post-paste edits from the Google monorepo and the associated comprehensive log. Because the quality of this data is critical, we devised a simple set of heuristics that constrained the notion of post-paste adjustments only to those in close proximity to the original paste location. With these heuristics, we extracted a training dataset from the prior few months and manually evaluated generated examples, iterating until the optimal state of the extraction heuristics was achieved. During this process, we noticed that 28% of all pastes have no follow-up edits (with variance among programming languages from 23% to 41%). We kept these examples in the training dataset, so the model could learn to output “no edits needed” as well.

We used a model pre-trained on coding-related tasks from DIDACT, and fine-tuned it using the annotated data set.

To keep developers in flow, the model had to be both fast and accurate. It needed to strike the right balance between suggesting entirely accurate edits and showing speculative suggestions that might provide some value but require follow-up changes.

Our exploration started with an arbitrarily chosen baseline of a model tuned to 65% exact-match precision, i.e., percentage of suggestions exactly matching the expected target from our evaluation set. Iterating on models of different sizes and different properties in offline experimentation, we only achieved marginal quality improvements at the cost of noticeable latency increases. After multiple iterations of experimentation and refinement of training data, we settled on a latency of ~150 ms, which resulted in the model showing a suggestion in 56% of the evaluation cases.

Our goal was to design a simple interaction model that would suggest high confidence edits in a clear and timely fashion. Suggestion confidence was set by a score threshold, acquired in an offline evaluation phase, set to exactly match 65% of the suggestions in the original training data. If a suggestion passed this threshold, it was displayed with decorative UI elements in the editor. To avoid interrupting the flow, suggestions were automatically discarded if the user did anything other than accepting the suggestion.

We rolled out the beta version of Smart Paste internally to about 3k engineers and observed their interaction behaviors. We noticed that in ~27% of the cases suggestions were discarded without the user even noticing them (i.e., less than 500 ms after showing them). To alleviate this, we added a window of 1.5 seconds after showing a suggestion, during which simple cursor movement would not discard it. We found that this allowed the user to notice the suggestion, without polluting the UI with outdated information. This change increased the acceptance rate by ~15%.

Initially, we had two UI patterns available in our IDE to show the suggested edits: grayed-out text (similar to that used for code completion) or a full diff view. The former couldn’t display deletions, while the latter imposed a significant cognitive load on the user, causing unwanted disruption. So, we instead decided to explore various novel patterns to visualize the suggestions.

We considered automatically applying the suggested edits and highlighting the changes, enabling the user to review and potentially revert them.

Finally, we considered showing the inserted and removed snippets directly in the editor in the form of an inline diff, which the user had to accept explicitly via the established TAB shortcut (used for both inline code completion and IntelliSense).

In the future, we plan to revisit the Auto-apply mode for small high-confidence changes. Suggestions with fewer than 10 characters have nearly half the acceptance rate compared to suggestions containing 10 to 100 characters. We hypothesize that this may be because users add the suggested change themselves right after pasting (e.g., adding a missing semicolon after pasting a full line), making the suggestion obsolete.

The next step for SmartPaste is to explore the model’s capabilities in providing helpful suggestions outside of the copy-paste interactions. Inspired by the real world usage of the feature, e.g., to automatically adapt all references to a property after renaming and copy-pasting (as shown below), we plan to examine different triggering mechanisms and UX solutions.

In examination of user behavior across approximately 40k engineers, we found that 6.9% of all pastes in the IDE utilized Smart Paste, with a 42.5% acceptance rate, which saves significant developer effort.

We introduced Smart Paste, a novel feature that provides AI-generated, context-aware adjustments to pasted code. It’s now enabled by default in Google’s most used IDE, and Googlers accept hundreds of thousands of its suggestions per week.

This work is the work of many people from the Google Core Systems & Experiences team, Google Research and Google DeepMind. We would like to extend our deep gratitude to Alexander Frömmgen as one of the key contributors and drivers of this work, for all his contributions, guidance, and helpful advice, which has helped to ensure that Smart Paste succeeds. We would also like to thank Guilherme Herzog for all his contributions to bringing the feature to a wider audience. We would like to also thank Kristóf Molnár for his expertise and help in bringing this post to a wider audience. This work couldn’t have happened without the great work and consultation from the DIDACT effort and contributions from all of our team members Adam Husting, Alberto Elizondo, Ambar Murillo, Boris Bokowski, Brett Durrett, Chris Gorgolewski, Damien Martin-Guillerez, David Tattersall, David Zhang, Gabriela Surita, Henrik Muehe, Iris Chu, Juanjo Carin, Kevin Villela, Madhura Dudhgaonkar, Maxim Tabachnyk, Nimesh Ghelani, Niranjan Tulpule, Pierre-Antoine Manzagol, Satish Chandra, Siddhant Sanyam, Stoyan Nikolov, Ugam Kumar, Vahid Meimand, Vincent Nguyen, and Zoubin Ghahramani. Thanks to Tom Small and Delphine Carlson for creating graphics for this post.