- Afshin Rostamizadeh
- Ameet Talwalkar
- Giulia DeSalvo
- Kevin Jamieson
- Lisha Li
Abstract
Performance of machine learning algorithms depends critically on identifying a good set of hyperparameters. While recent approaches use Bayesian Optimiza- tion to adaptively select configurations, we focus on speeding up random search through adaptive resource allocation. We present H YPERBAND , a novel algorithm for hyperparameter optimization that is simple, flexible, and theoretically sound. H YPERBAND is a principled early-stoppping method that adaptively allocates a pre- defined resource, e.g., iterations, data samples or number of features, to randomly sampled configurations. We compare H YPERBAND with state-of-the-art Bayesian Optimization methods on several hyperparameter optimization problems. We ob- serve that H YPERBAND can provide over an order of magnitude speedups over competitors on a variety of neural network and kernel-based learning problems.
Research Areas
Learn more about how we do research
We maintain a portfolio of research projects, providing individuals and teams the freedom to emphasize specific types of work