Customizing an LLM for Enterprise Software Engineering
Abstract
Software is built over time through small additions, revisions, deployments, maintenance, bug fixing, etc. These activities generate valuable data that modern LLMs could be finetuned on, to unlock additional tool possibilities for enterprise software engineering. While frontier LLMs are already very capable, this form of customization offers a compelling path for enterprise-specific optimization.
We introduce Gemini for Google(formerly Goose), an adaptation of Gemini specialized for Google's internal software engineering ecosystem. This paper details the model's end-to-end development, from curating a trillion-token proprietary dataset to implementing a mid-training strategy that mitigates catastrophic forgetting. In a large-scale blind A/B study across 29,000 developers, Gemini for Google significantly outperformed baselines: reducing the mean number of iterations per turn by 23%, and increasing code survival rates by 16.8%. Beyond metrics, we provide a comprehensive blueprint for enterprise model adaptation, covering: (1) the extraction of high-value signals from software engineering data, (2) data preparation strategies, (3) full-stack model tuning (pre- and post-training), and (4) the deployment of downstream applications. We believe this methodology offers a replicable path for other organizations to unlock the full potential of their internal engineering data.
We introduce Gemini for Google(formerly Goose), an adaptation of Gemini specialized for Google's internal software engineering ecosystem. This paper details the model's end-to-end development, from curating a trillion-token proprietary dataset to implementing a mid-training strategy that mitigates catastrophic forgetting. In a large-scale blind A/B study across 29,000 developers, Gemini for Google significantly outperformed baselines: reducing the mean number of iterations per turn by 23%, and increasing code survival rates by 16.8%. Beyond metrics, we provide a comprehensive blueprint for enterprise model adaptation, covering: (1) the extraction of high-value signals from software engineering data, (2) data preparation strategies, (3) full-stack model tuning (pre- and post-training), and (4) the deployment of downstream applications. We believe this methodology offers a replicable path for other organizations to unlock the full potential of their internal engineering data.
