Parallel Boosting with Momentum

Indraneel Mukherjee
Rafael Frongillo
Yoram Singer
ECML PKDD 2013, Part III, LNAI 8190, Springer, Heidelberg, pp. 17-32 (to appear)

Abstract

We describe a new, simplified, and general analysis of a fusion of Nesterov’s accelerated gradient with parallel coordinate descent. The resulting algorithm, which we call BOOM, for boosting with momentum, enjoys the merits of both techniques. Namely, BOOM retains the momentum and convergence properties of the accelerated gradient method while taking into account the curvature of the objective function. We describe a distributed implementation of BOOM which is suitable for massive high dimensional datasets. We show experimentally that BOOM is especially effective in large scale learning problems with rare yet informative features.

Research Areas