Black-Box Optimization of Unimodal Functions

Abhimanyu Das
Ashok Cutkosky
Weihao Kong
UAI 2023(2023)

Abstract

We provide an intuitive new algorithm for black-box stochastic optimization of unimodal functions, a function class that we observe empirically can capture hyperparameter-tuning loss surfaces. Our method's convergence guarantee automatically adapts to Lipschitz constants and other problem difficulty parameters, recovering and extending prior results. We complement our theoretical development with experimentally validation on hyperparameter tuning tasks.

Research Areas