Vector-based Navigation using Grid-like Representations in Artificial Agents.
Abstract
Efficient navigation is a fundamental component of mammalian behaviour but remains challenging for artificial agents. Mammalian spatial behaviour is underpinned by grid cells in the entorhinal cortex, providing a multi-scale periodic representation that functions as a metric for coding space. Grid cells are viewed as critical for integrating self-motion (path integration) and planning direct trajectories to goals (vector-based navigation).We report, for the first time, that brain-like grid representations can emerge as the product of optimizing a recurrent network to perform the task of path integration - providing a normative perspective on the role of grid cells as a compact code for representing space. We show that grid cells provide an effective basis set to optimize the primary objective of navigation through deep reinforcement learning (RL) - the rapid discovery and exploitation of goals in complex, unfamiliar, and changeable environments. The performance of agents endowed with grid-like representations was found to surpass that of an expert human and comparison agents. Further, we demonstrate that grid-like representations enable agents to conduct shortcut behaviours reminiscent of those performed by mammals - with decoding analyses confirming that the metric quantities necessary for vector-based navigation (e.g. Euclidean distance and direction to goal) are represented within the network. Our findings show that emergent grid-like responses furnish agents with a Euclidean spatial metric and associated vector operations, providing a foundation for proficient navigation. As such, our results support neuroscientific theories that see grid cells as critical for path integration and vector-based navigation, demonstrating that the latter can be combined with path- based strategies to support navigation in complex environments
Research Areas
