From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Following
Abstract
Reinforcement learning is a promising framework for solving control problems,
but its use in practical situations is hampered by the fact that reward functions are
often difficult to engineer. Specifying goals and tasks for autonomous machines,
such as robots, is a significant challenge: conventionally, reward functions and
goal states have been used to communicate objectives. But people can communicate objectives to each other simply by describing or demonstrating them. How
can we build learning algorithms that will allow us to tell machines what we want
them to do? In this work, we investigate the problem of grounding language commands as reward functions using inverse reinforcement learning, and argue that
language-conditioned rewards are more transferable than language-conditioned
policies to new environments. We propose language-conditioned reward learning
(LC-RL), which grounds language commands as a reward function represented by
a deep neural network. We demonstrate that our model learns rewards that transfer to novel tasks and environments on realistic, high-dimensional visual environments with natural language commands, whereas directly learning a languageconditioned policy leads to poor performance.
but its use in practical situations is hampered by the fact that reward functions are
often difficult to engineer. Specifying goals and tasks for autonomous machines,
such as robots, is a significant challenge: conventionally, reward functions and
goal states have been used to communicate objectives. But people can communicate objectives to each other simply by describing or demonstrating them. How
can we build learning algorithms that will allow us to tell machines what we want
them to do? In this work, we investigate the problem of grounding language commands as reward functions using inverse reinforcement learning, and argue that
language-conditioned rewards are more transferable than language-conditioned
policies to new environments. We propose language-conditioned reward learning
(LC-RL), which grounds language commands as a reward function represented by
a deep neural network. We demonstrate that our model learns rewards that transfer to novel tasks and environments on realistic, high-dimensional visual environments with natural language commands, whereas directly learning a languageconditioned policy leads to poor performance.
Research Areas
