Web-Scale Job Scheduling

Web datacenters and clusters can be larger than the world’s largest supercomputers, and run workloads that are at least as heteroge- neous and complex as their high-performance computing counterparts. And yet little is known about the unique job scheduling challenges of these environments. This article aims to ameliorate this situation. It dis- cusses the challenges of running web infrastructure and describes several techniques to address them. It also presents some of the problems that remain open in the field.