Google Research

Scalable Build Service System with Smart Scheduling Service

  • Ahmed Mustafa Ali Gad
  • Daniel Lucas Rall
  • Kaiyuan Wang
  • Vijay Sagar Gullapalli
  • Xin Huang
International Symposium on Software Testing and Analysis (2020) (to appear)


Build automation is critical for developers to check if their code compiles, passes all tests and is able to deploy to the server. Many companies adopt Continuous Integration (CI) services to make sure that the code changes from multiple developers can be safely merged at the head of the project. Internally, CI triggers software builds to makes sure that the new code change does not break the compila- tion or the tests. For any large company which has a monolithic code repository and thousands of developers, it is hard to make sure that all code changes are safe to submit in a timely manner. Because each code change may involve multiple builds, and there could be millions of builds to run each day to guarantee developers’ daily productivity. Company C is one of those large companies that need a scalable build service to support developers’ work. More than 100,000 code changes are submitted to our repository on average each day, in- cluding changes from either human users or automated tools. More than 15 million builds are executed on average each day. In this EXPERIENCE paper, we first describe an overview of our scal- able build service architecture. Then, we discuss more details about how we make build scheduling decisions. Finally, we discuss some experience in the scalability of the build service system and the performance of the build scheduling service.

Research Areas

Learn more about how we do research

We maintain a portfolio of research projects, providing individuals and teams the freedom to emphasize specific types of work