Jesse Farebrother
Jesse Farebrother is a Research Scientist at Google DeepMind in London, focusing on foundational AI research. His research interests lie at the intersection of deep learning and reinforcement learning, specifically leveraging unsupervised pre-training to learn foundation models of behavior with the goal of enabling the real-world deployment of intelligent decision-making agents. He received his Ph.D. in Computer Science from McGill University and Mila, where he was supervised by Marc G. Bellemare and Doina Precup. During his doctoral studies, he interned at Meta FAIR in Paris and Montréal, and previously spent time as a Student Researcher at Google DeepMind and Google Brain. Prior to his graduate studies, Jesse completed his B.Sc. in Computing Science at the University of Alberta, and served as the lead developer and maintainer of the Arcade Learning Environment (ALE). More information about his research and publications can be found here.
Research Areas
Authored Publications
Sort By
Proto-Value Networks: Scaling Representation Learning with Auxiliary Tasks
Joshua Greaves
Rishabh Agarwal
Charline Le Lan
Marc Bellemare
International Conference on Learning Representations (ICLR) (2023)
Preview abstract
Auxiliary tasks improve the representations learned by deep reinforcement learning agents. Analytically, their effect is reasonably well-understood; in practice, how-ever, their primary use remains in support of a main learning objective, rather than as a method for learning representations. This is perhaps surprising given that many auxiliary tasks are defined procedurally, and hence can be treated as an essentially infinite source of information about the environment. Based on this observation, we study the effectiveness of auxiliary tasks for learning rich representations, focusing on the setting where the number of tasks and the size of the agent’s network are simultaneously increased. For this purpose, we derive a new family of auxiliary tasks based on the successor measure. These tasks are easy to implement and have appealing theoretical properties. Combined with a suitable off-policy learning rule, the result is a representation learning algorithm that can be understood as extending Mahadevan & Maggioni (2007)’s proto-value functions to deep reinforcement learning – accordingly, we call the resulting object proto-value networks. Through a series of experiments on the Arcade Learning Environment, we demonstrate that proto-value networks produce rich features that may be used to obtain performance comparable to established algorithms, using only linear approximation and a small number (~4M) of interactions with the environment’s reward function.
View details
