Scalable Build Service System with Smart Scheduling Service
Abstract
Build automation is critical for developers to check if their code
compiles, passes all tests and is able to deploy to the server. Many
companies adopt Continuous Integration (CI) services to make sure
that the code changes from multiple developers can be safely merged
at the head of the project. Internally, CI triggers software builds to
makes sure that the new code change does not break the compila-
tion or the tests. For any large company which has a monolithic
code repository and thousands of developers, it is hard to make
sure that all code changes are safe to submit in a timely manner.
Because each code change may involve multiple builds, and there
could be millions of builds to run each day to guarantee developers’
daily productivity.
Company C is one of those large companies that need a scalable
build service to support developers’ work. More than 100,000 code
changes are submitted to our repository on average each day, in-
cluding changes from either human users or automated tools. More
than 15 million builds are executed on average each day. In this
EXPERIENCE paper, we first describe an overview of our scal-
able build service architecture. Then, we discuss more details about
how we make build scheduling decisions. Finally, we discuss some
experience in the scalability of the build service system and the
performance of the build scheduling service.
compiles, passes all tests and is able to deploy to the server. Many
companies adopt Continuous Integration (CI) services to make sure
that the code changes from multiple developers can be safely merged
at the head of the project. Internally, CI triggers software builds to
makes sure that the new code change does not break the compila-
tion or the tests. For any large company which has a monolithic
code repository and thousands of developers, it is hard to make
sure that all code changes are safe to submit in a timely manner.
Because each code change may involve multiple builds, and there
could be millions of builds to run each day to guarantee developers’
daily productivity.
Company C is one of those large companies that need a scalable
build service to support developers’ work. More than 100,000 code
changes are submitted to our repository on average each day, in-
cluding changes from either human users or automated tools. More
than 15 million builds are executed on average each day. In this
EXPERIENCE paper, we first describe an overview of our scal-
able build service architecture. Then, we discuss more details about
how we make build scheduling decisions. Finally, we discuss some
experience in the scalability of the build service system and the
performance of the build scheduling service.
Research Areas
