Google Research

Vector-based Navigation using Grid-like Representations in Artificial Agents.

  • Alexander Pritzel
  • Andrea Banino
  • Benigno Uria
  • Brian C Zhang
  • Caswell Barry
  • Charles Blundell
  • Charlie Beattie
  • Demis Hassabis
  • Dharshan Kumaran
  • Fabio Viola
  • Greg Wayne
  • Helen King
  • Hubert Soyer
  • Joseph Modayil
  • Koray Kavukcuoglu
  • Martin J. Chadwick
  • Neil Rabinowitz
  • Piotr Mirowski
  • Raia Hadsell
  • Razvan Pascanu
  • Ross Goroshin
  • Stephen Gaffney
  • Stig Vilholm Petersen
  • Thomas Degris
  • Timothy Lillicrap
Nature (2018)

Abstract

Efficient navigation is a fundamental component of mammalian behaviour but remains challenging for artificial agents. Mammalian spatial behaviour is underpinned by grid cells in the entorhinal cortex, providing a multi-scale periodic representation that functions as a metric for coding space. Grid cells are viewed as critical for integrating self-motion (path integration) and planning direct trajectories to goals (vector-based navigation).We report, for the first time, that brain-like grid representations can emerge as the product of optimizing a recurrent network to perform the task of path integration - providing a normative perspective on the role of grid cells as a compact code for representing space. We show that grid cells provide an effective basis set to optimize the primary objective of navigation through deep reinforcement learning (RL) - the rapid discovery and exploitation of goals in complex, unfamiliar, and changeable environments. The performance of agents endowed with grid-like representations was found to surpass that of an expert human and comparison agents. Further, we demonstrate that grid-like representations enable agents to conduct shortcut behaviours reminiscent of those performed by mammals - with decoding analyses confirming that the metric quantities necessary for vector-based navigation (e.g. Euclidean distance and direction to goal) are represented within the network. Our findings show that emergent grid-like responses furnish agents with a Euclidean spatial metric and associated vector operations, providing a foundation for proficient navigation. As such, our results support neuroscientific theories that see grid cells as critical for path integration and vector-based navigation, demonstrating that the latter can be combined with path- based strategies to support navigation in complex environments

Research Areas

Learn more about how we do research

We maintain a portfolio of research projects, providing individuals and teams the freedom to emphasize specific types of work