Challenges of Real-World Reinforcement Learning:Definitions, Benchmarks & Analysis

Reinforcement learning (RL) has proven its worth in a series of artificialdomains, and is beginning to show some successes in real-world scenarios. However,much of the research advances in RL are hard to leverage in real-world systemsdue to a series of assumptions that are rarely satisfied in practice. We identifyand formalize a series of independent challenges that embody the difficulties thatmust be addressed for RL to be commonly deployed in real-world systems. Foreach challenge, we define it formally in the context of a Markov Decision Process,analyze the effects of the challenge on state-of-the-art learning algorithms, andpresent some existing attempts at tackling it. We believe that an approach thataddresses all nine challenges would be readily deployable in a large number of realworld problems. We implement our proposed challenges in a suite of continuouscontrol environments calledrealworldrl-suitewhich we propose an as an open-source benchmark.

• Machine Intelligence