Efficient Hyperparameter Optimization and Infinitely Many Armed Bandits

Performance of machine learning algorithms depends critically on identifying a
good set of hyperparameters. While recent approaches use Bayesian Optimiza-
tion to adaptively select configurations, we focus on speeding up random search
through adaptive resource allocation. We present H YPERBAND , a novel algorithm
for hyperparameter optimization that is simple, flexible, and theoretically sound.
H YPERBAND is a principled early-stoppping method that adaptively allocates a pre-
defined resource, e.g., iterations, data samples or number of features, to randomly
sampled configurations. We compare H YPERBAND with state-of-the-art Bayesian
Optimization methods on several hyperparameter optimization problems. We ob-
serve that H YPERBAND can provide over an order of magnitude speedups over
competitors on a variety of neural network and kernel-based learning problems.

• Machine Intelligence