Robotic Table Tennis: A Case Study into a High Speed Learning System

Barney J. Reed
Peng Xu
Erwin Johan Coumans
Satoshi Kataoka
Corey Lynch
Navdeep Jaitly
Anish Shankar
Grace Vesom
Yuheng Kuang
Ken Oslund
Thinh Nguyen
Gus Kouretas
Saminda Abeyruwan
Juhana Kangaspunta
Justin Boyd
Michael Ahn
Wenbo Gao
Omar Escareno
Avi Singh
Jon Abelian
Robotics: Science and Systems (2023)

Abstract

We present a deep-dive into a learning robotic system that, in previous work, was shown to be capable of hundreds of table tennis rallies with a human and has the ability to precisely return the ball to desired targets. This system puts together a highly optimized and novel perception subsystem, a high-speed low-latency robot controller, a simulation paradigm that can prevent damage in the real world and also train policies for zero-shot transfer, and automated real world environment resets that enable autonomous training and evaluation on physical robots. We complement a complete system description including numerous design decisions that are typically not widely disseminated, with a collection of ablation studies that clarify the importance of mitigating various sources of latency, accounting for training and deployment distribution shifts, robustness of the perception system, and sensitivity to policy hyper-parameters and choice of action space. A video demonstrating the components of our system and details of experimental results is included in the supplementary material.