Hossein Mobahi
I am a Research Scientist at Google DeepMind. I joined Google Research in 2016. Before that, I was a Postdoctoral Researcher at the Computer Science and Artificial Intelligence Laboratory (CSAIL) at MIT, where I had the privilege of working with Professor Bill Freeman and Dr. John Fisher. I earned my PhD in Computer Science from the University of Illinois at Urbana-Champaign (UIUC), where I was fortunate to be supervised by Professor Yi Ma.
My broad interest lies in Artificial Intelligence, with a specific focus on the intersection of Machine Learning and Optimization. My research is often guided by mathematical principles, aiming to develop practically successful methods that offer greater clarity in their understanding or improved performance.
I am a co-creator of the Sharpness Aware Minimization (SAM) method. My work also includes contributions to the theory of self-distillation, optimization by the continuation method, and understanding the loss surface of neural networks. I have a long-standing interest in radically new approaches to creating neural architectures, particularly those inspired by biology and the human brain, as well as curriculum learning strategies that aim to enhance training efficiency and generalization through more effective data presentation.
Research Areas
Authored Publications
Sort By
Google
Sharpness-Aware Minimization Improves Language Model Generalization
Yi Tay
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (2022), pp. 7360-7371
Methods and Analysis of The First Competition in Predicting Generalization of Deep Learning
Yiding Jiang
Parth Natekar
Manik Sharma
Sumukh K. Aithal
Dhruva Kashyap
Natarajan Subramanyam
Carlos Lassance
Daniel M. Roy
Gintare Karolina Dziugaite
Suriya Gunasekar
Isabelle Guyon
Pierre Foret
Scott Yak i
Behnam Neyshabur
Samy Bengio
Proceedings of the NeurIPS 2020 Competition and Demonstration Track, PMLR (2021)
NeurIPS 2020 Competition: Predicting Generalization in Deep Learning
Yiding Jiang
Pierre Foret
Scott Yak
Daniel M. Roy
Gintare Karolina Dziugaite
Samy Bengio
Suriya Gunasekar
Isabelle Guyon
Behnam Neyshabur
arXiv (2020)
Self-Distillation Amplifies Regularization in Hilbert Space
Mehrdad Farajtabar
Peter Bartlett
Neural Information Processing Systems (NeurIPS) (2020)
