Google Research

Depth creates no more spurious local minima

(2019)

Abstract

We show that for any convex differentiable loss function, a deep linear network has no spurious local minima as long as it is true for the two layer case. When applied to the quadratic loss, our result immediately implies the powerful result in [Kawaguchi 2016] that there is no spurious local minima in deep linear networks. Further, with the recent work [Zhou and Liang 2018], we can remove all the assumptions in [Kawaguchi 2016]. Our proof is short and elementary. It builds on the recent work of [Laurent and von Brecht 2018] and uses a new rank one perturbation argument.

Research Areas

Learn more about how we do research

We maintain a portfolio of research projects, providing individuals and teams the freedom to emphasize specific types of work