Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Google’s Datacenter Network

Amin Vahdat
Gaurav Desai
Joon Ong
Ashby Armistead
Jason Simmons
Seb Boving
Amit Agarwal
Eiichi Tanda
Glen Anderson
Jim Wanderer
Roy Bannon
Jeff Provost
Paulie Germano
Sigcomm '15, Google Inc (2015)
Google Scholar

Abstract

We present our approach for overcoming the cost, operational complexity, and limited scale endemic to datacenter networks a decade ago. Three themes unify the five generations of datacenter networks detailed in this paper. First, multi-stage Clos topologies built from commodity switch silicon can support cost-effective deployment of building-scale networks. Second, much of the general, but complex, decentralized network routing and management protocols supporting arbitrary deployment scenarios were overkill for single-operator, pre-planned datacenter networks. We built a centralized control mechanism based on a global configuration pushed to all datacenter switches. Third, modular hardware design coupled with simple, robust software allowed our design to also support inter-cluster and wide-area networks. Our datacenter networks run at dozens of sites across the planet, scaling in capacity by 100x over ten years to more than 1Pbps of bisection bandwidth.

Research Areas