Characterization and Comparison of Cloud versus Grid Workloads
Abstract
A new era of Cloud Computing has emerged, but
the characteristics of Cloud load in data centers is not perfectly
clear. Yet this characterization is critical for the design of novel
Cloud job and resource management systems. In this paper, we
comprehensively characterize the job/task load and host load
in a real-world production data center at Google Inc. We use
a detailed trace of over 25 million tasks across over 12,500
hosts. We study the differences between a Google data center
and other Grid/HPC systems, from the perspective of both work
load (w.r.t. jobs and tasks) and host load (w.r.t. machines). In
particular, we study the job length, job submission frequency,
and the resource utilization of jobs in the different systems,
and also investigate valuable statistics of machine’s maximum
load, queue state and relative usage levels, with different job
priorities and resource attributes. We find that the Google data
center exhibits finer resource allocation with respect to CPU
and memory than that of Grid/HPC systems. Google jobs are
always submitted with much higher frequency and they are
much shorter than Grid jobs. As such, Google host load exhibits
higher variance and noise.
the characteristics of Cloud load in data centers is not perfectly
clear. Yet this characterization is critical for the design of novel
Cloud job and resource management systems. In this paper, we
comprehensively characterize the job/task load and host load
in a real-world production data center at Google Inc. We use
a detailed trace of over 25 million tasks across over 12,500
hosts. We study the differences between a Google data center
and other Grid/HPC systems, from the perspective of both work
load (w.r.t. jobs and tasks) and host load (w.r.t. machines). In
particular, we study the job length, job submission frequency,
and the resource utilization of jobs in the different systems,
and also investigate valuable statistics of machine’s maximum
load, queue state and relative usage levels, with different job
priorities and resource attributes. We find that the Google data
center exhibits finer resource allocation with respect to CPU
and memory than that of Grid/HPC systems. Google jobs are
always submitted with much higher frequency and they are
much shorter than Grid jobs. As such, Google host load exhibits
higher variance and noise.
Research Areas
