# Learning to Play Table Tennis From Scratch using Muscular Robots

arXiv: https://arxiv.org/abs/2006.05935 · HTML: https://mlsanity.com/paper/2006.05935/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2020.

## Built on by (most cited first, 30 of 35)

- [Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes](https://mlsanity.com/paper/2408.03539/) (2024)
- [SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning](https://mlsanity.com/paper/2401.16013/) (2024)
- [Creating a Dynamic Quadrupedal Robotic Goalkeeper with Reinforcement Learning](https://mlsanity.com/paper/2210.04435/) (2022)
- [Robotic Table Tennis: A Case Study into a High Speed Learning System](https://mlsanity.com/paper/2309.03315/) (2023)
- [Fast Kinodynamic Planning on the Constraint Manifold with Deep Neural Networks](https://mlsanity.com/paper/2301.04330/) (2023)
- [Hidden Parameter Recurrent State Space Models For Changing Dynamics Scenarios](https://mlsanity.com/paper/2206.14697/) (2022)
- [SpinDOE: A ball spin estimation method for table tennis robot](https://mlsanity.com/paper/2303.03879/) (2023)
- [Stochastic Online Optimization for Cyber-Physical and Robotic Systems](https://mlsanity.com/paper/2404.05318/) (2024)
- [Contextualized Hybrid Ensemble Q-learning: Learning Fast with Control Priors](https://mlsanity.com/paper/2406.19768/) (2024)
- [Learning Diverse Robot Striking Motions with Diffusion Models and Kinematically Constrained Gradient Guidance](https://mlsanity.com/paper/2409.15528/) (2024)
- [LATTE-MV: Learning to Anticipate Table Tennis Hits from Monocular Videos](https://mlsanity.com/paper/2503.20936/) (2025)
- [RP1M: A Large-Scale Motion Dataset for Piano Playing with Bi-Manual Dexterous Robot Hands](https://mlsanity.com/paper/2408.11048/) (2024)
- [Bridging the gap between Learning-to-plan, Motion Primitives and Safe Reinforcement Learning](https://mlsanity.com/paper/2408.14063/) (2024)
- [Integrating Learning-Based Manipulation and Physics-Based Locomotion for Whole-Body Badminton Robot Control](https://mlsanity.com/paper/2504.17771/) (2025)
- [Learning Human-Like Badminton Skills for Humanoid Robots](https://mlsanity.com/paper/2602.08370/) (2026)
- [Hindsight States: Blending Sim and Real Task Elements for Efficient Reinforcement Learning](https://mlsanity.com/paper/2303.02234/) (2023)
- [Toward Scalable Multirobot Control: Fast Policy Learning in Distributed MPC](https://mlsanity.com/paper/2412.19669/) (2024)
- [PACE: Physics Augmentation for Coordinated End-to-end Reinforcement Learning toward Versatile Humanoid Table Tennis](https://mlsanity.com/paper/2509.21690/) (2025)
- [Learning to Exploit Elastic Actuators for Quadruped Locomotion](https://mlsanity.com/paper/2209.07171/) (2022)
- [Off-Policy Deep Reinforcement Learning Algorithms for Handling Various Robotic Manipulator Tasks](https://mlsanity.com/paper/2212.05572/) (2022)
- [A multi-modal table tennis robot system](https://mlsanity.com/paper/2310.19062/) (2023)
- [Learning World Models With Hierarchical Temporal Abstractions: A Probabilistic Perspective](https://mlsanity.com/paper/2404.16078/) (2024)
- [A Retrospective on the Robot Air Hockey Challenge: Benchmarking Robust, Reliable, and Safe Learning Techniques for Real-world Robotics](https://mlsanity.com/paper/2411.05718/) (2024)
- [CHEQ-ing the Box: Safe Variable Impedance Learning for Robotic Polishing](https://mlsanity.com/paper/2501.07985/) (2025)
- [Learning Nonlinear Causal Reductions to Explain Reinforcement Learning Policies](https://mlsanity.com/paper/2507.14901/) (2025)
- [Efficient Model-Based Reinforcement Learning for Robot Control via Online Optimization](https://mlsanity.com/paper/2510.18518/) (2025)
- [Position: RL Researchers Need to Distinguish Between Solving Simulators and Using Simulators as a Proxy](https://mlsanity.com/paper/2606.28433/) (2026)
- [Object Manipulation in Marine Environments using Reinforcement Learning](https://mlsanity.com/paper/2406.03223/) (2024)
- [Catching Spinning Table Tennis Balls in Simulation with End-to-End Curriculum Reinforcement Learning](https://mlsanity.com/paper/2503.01251/) (2025)
- [SpikePingpong: Spike Vision-based Fast-Slow Pingpong Robot System](https://mlsanity.com/paper/2506.06690/) (2025)

## Abstract (arXiv)

Dynamic tasks like table tennis are relatively easy to learn for humans but pose significant challenges to robots. Such tasks require accurate control of fast movements and precise timing in the presence of imprecise state estimation of the flying ball and the robot. Reinforcement Learning (RL) has shown promise in learning of complex control tasks from data. However, applying step-based RL to dynamic tasks on real systems is safety-critical as RL requires exploring and failing safely for millions of time steps in high-speed regimes. In this paper, we demonstrate that safe learning of table tennis using model-free Reinforcement Learning can be achieved by using robot arms driven by pneumatic artificial muscles (PAMs). Softness and back-drivability properties of PAMs prevent the system from leaving the safe region of its state space. In this manner, RL empowers the robot to return and smash real balls with 5 ms and 12ms on average to a desired landing point. Our setup allows the agent to learn this safety-critical task (i) without safety constraints in the algorithm, (ii) while maximizing the speed of returned balls directly in the reward function (iii) using a stochastic policy that acts directly on the low-level controls of the real system and (iv) trains for thousands of trials (v) from scratch without any prior knowledge. Additionally, we present HYSR, a practical hybrid sim and real training that avoids playing real balls during training by randomly replaying recorded ball trajectories in simulation and applying actions to the real robot. This work is the first to (a) fail-safe learn of a safety-critical dynamic task using anthropomorphic robot arms, (b) learn a precision-demanding problem with a PAM-driven system despite the control challenges and (c) train robots to play table tennis without real balls. Videos and datasets are available at muscularTT.embodied.ml.