Arjun Singh
Arjun is an Engineering Fellow and Technical Lead for networking in Google. During his tenure in Google, he has worked on developing solutions for Google’s data center, Wide Area and Edge/Peering networks with a focus on Software-defined networking. Arjun has participated in over five generations of data center and wide area networking infrastructure at Google over 19 years and has been recognized with the ACM SIGCOMM Networking Systems Award and Test of Time Paper Award for his work. Before joining Google, Arjun received a PhD, M.S. in Electrical Engineering from Stanford University and a Bachelor of Technology in Computer Science and Engineering from the Indian Institute of Technology (IIT), Kharagpur.
Research Areas
Authored Publications
Sort By
CAPA: An Architecture For Operating Cluster Networks With High Availability
Bingzhe Liu
Brighten Godfrey
Omid Alipourfard
Joon Ong
Virginia Beauregard
Mukarram Tariq
Mayur Patel
Prerepa Viswanadham
Manish Verma
Xander Lin
Patrick Conner
Deepak Arulkannan
Amr Sabaa
Rich Alimi
Alex Smirnov
Google, Google, 1600 Amphitheatre Pkwy, Mountain View, CA 94043 (2023)
Preview abstract
Management operations are a major source of outages for networks. A number of best practices designed to reduce and mitigate such outages are well known, but their enforcement has been challenging, leaving the network vulnerable to inadvertent mistakes and gaps which repeatedly result in outages. We present our experiences with CAPA, Google’s “containment and prevention architecture” for regulating management operations on our cluster networking fleet. Our goal with CAPA is to limit the systems where strict adherence to best practices is required, so that availability of the network is not dependent on the good intentions of every engineer and operator. We enumerate the features of CAPA which we have found to be necessary to effectively enforce best practices within a thin “regulation“ layer. We evaluate CAPA based on case studies of outages prevented, counterfactual analysis of past incidents, and known limitations. Management-plane-related outages have substantially reduced both in frequency and severity, with a 82% reduction in cumulative duration of incidents normalized to fleet size over five years
View details
Aquila: A unified, low-latency fabric for datacenter networks
Zuowei Shen
Moray Mclaren
Raghuraman Balasubramanian
Michael Cutforth
Sunghwan Yoo
Hema Hariharan
David Decotigny
Peter James Cuy
Eric Lance
Joe Zbiciak
Rakesh Gautam
Ming Tan
Zhehua Wu
Ye Tang
Stephen Wang
Rick Roy
Monica C Wong-Chan
Prashant Chandra
Aquila: A unified, low-latency fabric for datacenter networks (2022)
Preview abstract
Datacenter workloads have evolved from the data intensive, loosely-coupled workloads of the past decade to more tightly coupled ones, wherein ultra-low latency communication is essential for resource disaggregation over the network and to enable emerging programming models.
We introduce Aquila, an experimental datacenter network fabric built with ultra-low latency support as a first-class design goal, while also supporting traditional datacenter traffic. Aquila uses a new Layer 2 cell-based protocol, GNet, an integrated switch, and a custom ASIC with low-latency Remote Memory Access (RMA) capabilities co-designed with GNet. We demonstrate that Aquila is able to achieve under 40 μs tail fabric Round Trip Time (RTT) for IP traffic and sub-10 μs RMA execution time across hundreds of host machines, even in the presence of background throughput-oriented IP traffic. This translates to more than 5x reduction in tail latency for a production quality key-value store running on a prototype Aquila network.
View details
Orion: Google’s Software-Defined Networking Control Plane
Shawn Chen
Lorenzo Vicisano
Karthik Swaminathan Nagaraj
Shidong Zhang
Joon Suan Ong
Min Zhu
Mike Conley
Waqar Mohsin
Henrik Muehe
Amr Sabaa
KondapaNaidu Bollineni
Rich Alimi
(2021)
Preview abstract
We present Orion, a distributed Software-Defined Networking platform deployed globally in Google’s datacenter (Jupiter) as well as Wide Area (B4) networks. Orion was designed around a modular, micro-service architecture with a central publish-subscribe database to enable a distributed, yet tightly-coupled, software-defined network control system. Orion enables intent-based management and control, is highly scalable and amenable to global control hierarchies.
Over the years, Orion has matured with continuously improving performance in convergence (up to 40x faster), throughput (handling up to 1.16 million network updates per second), system scalability (supporting 16x larger networks), and data plane availability (50x, 100x reduction in unavailable time in Jupiter and B4, respectively) while maintaining high development velocity with bi-weekly release cadence. Today, Orion robustly enables all of Google’s Software-Defined Networks defending against failure modes that are both generic to large scale production networks as well as unique to SDN systems.
View details
Taking the Edge off with Espresso: Scale, Reliability and Programmability for Global Internet Peering
Calvin Ying
TaeEun Kim
Matthew Holliman
Ashok Narayanan
Colin Rice
Puneet Sood
Mukarram Tariq
Gary Baldus
Dzevad Trumic
Victor Lin
Bert Tanaka
Manish Verma
Brian Rogan
Vytautas Valancius
Mahesh Kallahalla
Marcus Hines
Sigcomm (2017)
Preview abstract
We present the design of Espresso, Google’s SDN-based Internet peering edge routing infrastructure. This architecture grew out of a need to exponentially scale the Internet edge cost-effectively and to
enable application-aware routing at Internet-peering scale. Espresso utilizes commodity switches and host-based routing/packet processing to implement a novel fine-grained traffic engineering capability.
Overall, Espresso provides Google a scalable peering edge that is programmable, reliable, and integrated with global traffic systems. Espresso also greatly accelerated deployment of new networking features at our peering edge. Espresso has been in production for two years and serves over 22% of Google’s total traffic to the Internet.
View details
Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Google's Datacenter Network
Eiichi Tanda
Amit Agarwal
Bob Felderman
Anand Kanagala
Seb Boving
Jim Wanderer
Ashby Armistead
Jason Simmons
Gaurav Desai
Jeff Provost
Roy Bannon
Joon Ong
Paulie Germano
Glen Anderson
Stephen Stuart
Communications of the ACM, Vol. 59, No. 9 (2016), pp. 88-97
Preview abstract
We present our approach for overcoming the cost, operational complexity, and limited scale endemic to datacenter networks a decade ago. Three themes unify the five generations of datacenter networks detailed in this paper. First, multi-stage Clos topologies built from commodity switch silicon can support cost-effective deployment of building-scale networks. Second, much of the general, but complex, decentralized network routing and management protocols supporting arbitrary deployment scenarios were overkill for single-operator, pre-planned datacenter networks. We built a centralized control mechanism based on a global configuration pushed to all datacenter switches. Third, modular hardware design coupled with simple, robust software allowed our design to also support inter-cluster and wide-area networks. Our datacenter networks run at dozens of sites across the planet, scaling in capacity by 100x over 10 years to more than 1 Pbps of bisection bandwidth.
View details
Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Google’s Datacenter Network
Amin Vahdat
Gaurav Desai
Joon Ong
Ashby Armistead
Jason Simmons
Seb Boving
Amit Agarwal
Eiichi Tanda
Glen Anderson
Jim Wanderer
Roy Bannon
Jeff Provost
Paulie Germano
Sigcomm '15, Google Inc (2015)
Preview abstract
We present our approach for overcoming the cost, operational complexity, and limited scale endemic to datacenter networks a decade ago. Three themes unify the five generations of datacenter networks detailed in this paper. First, multi-stage Clos topologies built from commodity switch silicon can support cost-effective deployment of building-scale networks. Second, much of the general, but complex, decentralized network routing and management protocols supporting arbitrary deployment scenarios were overkill for single-operator, pre-planned datacenter networks. We built a centralized control mechanism based on a global configuration pushed to all datacenter switches. Third, modular hardware design coupled with simple, robust software allowed our design to also support inter-cluster and wide-area networks. Our datacenter networks run at dozens of sites across the planet, scaling in capacity by 100x over ten years to more than 1Pbps of bisection bandwidth.
View details
WCMP: Weighted Cost Multipathing for Improved Fairness in Data Centers
Min Zhu
Abdul Kabbani
Malveeka Tewari
EuroSys '14: Proceedings of the Ninth European Conference on Computer Systems (2014), Article No. 5
Preview abstract
Data Center topologies employ multiple paths among servers to deliver scalable, cost-effective network capacity. The simplest and the most widely deployed approach for load balancing among these paths, Equal Cost Multipath (ECMP), hashes flows among the shortest paths toward a destination. ECMP leverages uniform hashing of balanced flow sizes to achieve fairness and good load balancing in data centers. However, we show that ECMP further assumes a balanced, regular, and fault-free topology, which are invalid assumptions in practice that can lead to substantial performance degradation and, worse, variation in flow bandwidths even for same size flows.
We present a set of simple algorithms that achieve Weighted Cost Multipath (WCMP) to balance traffic in the data center based on the changing network topology. The state required for WCMP is already disseminated as part of standard routing protocols and it can be readily implemented in the current switch silicon without any hardware modifications. We show how to deploy WCMP in a production OpenFlow network environment and present experimental and simulation results to show that variation in flow bandwidths can be reduced by as much as 25X by employing WCMP relative to ECMP.
View details
B4: Experience with a Globally Deployed Software Defined WAN
Preview
Junlan Zhou
Min Zhu
Amin Vahdat
Joon Ong
Subbaiah Venkata
Jim Wanderer
Sushant Jain
Proceedings of the ACM SIGCOMM Conference, Hong Kong, China (2013)
Preview abstract
One of the goals of traffic engineering is to achieve a
flexible trade-off between fairness and throughput so that users
are satisfied with their bandwidth allocation and the network
operator is satisfied with the utilization of network resources. In
this paper, we propose a novel way to balance the throughput
and fairness objectives with linear programming. It allows the
network operator to precisely control the trade-off by bounding
the fairness degradation for each commodity compared to the
max-min fair solution or the throughput degradation compared
to the optimal throughput. We also present improvements to a
previous algorithm that achieves max-min fairness by solving a
series of linear programs. We significantly reduce the number
of steps needed when the access rate of commodities is limited.
We extend the algorithm to two important practical use cases:
importance weights and piece-wise linear utility functions for
commodities. Our experiments on synthetic and real networks
show that our algorithms achieve a significant speedup and
provide practical insights on the trade-off between fairness and
throughput.
View details
