# Robotic Table Tennis with Model-Free Reinforcement Learning

arXiv: https://arxiv.org/abs/2003.14398 · HTML: https://mlsanity.com/paper/2003.14398/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Mar 2020.

## Built on by (most cited first, 15 of 15)

- [Deep Transformer Q-Networks for Partially Observable Reinforcement Learning](https://mlsanity.com/paper/2206.01078/) (2022)
- [Creating a Dynamic Quadrupedal Robotic Goalkeeper with Reinforcement Learning](https://mlsanity.com/paper/2210.04435/) (2022)
- [Provably Safe Deep Reinforcement Learning for Robotic Manipulation in Human Environments](https://mlsanity.com/paper/2205.06311/) (2022)
- [Fast Lifelong Adaptive Inverse Reinforcement Learning from Demonstrations](https://mlsanity.com/paper/2209.11908/) (2022)
- [Robotic Table Tennis: A Case Study into a High Speed Learning System](https://mlsanity.com/paper/2309.03315/) (2023)
- [Learning Diverse Robot Striking Motions with Diffusion Models and Kinematically Constrained Gradient Guidance](https://mlsanity.com/paper/2409.15528/) (2024)
- [LATTE-MV: Learning to Anticipate Table Tennis Hits from Monocular Videos](https://mlsanity.com/paper/2503.20936/) (2025)
- [Integrating Learning-Based Manipulation and Physics-Based Locomotion for Whole-Body Badminton Robot Control](https://mlsanity.com/paper/2504.17771/) (2025)
- [EnTRPO: Trust Region Policy Optimization Method with Entropy Regularization](https://mlsanity.com/paper/2110.13373/) (2021)
- [PACE: Physics Augmentation for Coordinated End-to-end Reinforcement Learning toward Versatile Humanoid Table Tennis](https://mlsanity.com/paper/2509.21690/) (2025)
- [Off-Policy Deep Reinforcement Learning Algorithms for Handling Various Robotic Manipulator Tasks](https://mlsanity.com/paper/2212.05572/) (2022)
- [A multi-modal table tennis robot system](https://mlsanity.com/paper/2310.19062/) (2023)
- [ES-ENAS: Efficient Evolutionary Optimization for Large Hybrid Search Spaces](https://mlsanity.com/paper/2101.07415/) (2021)
- [Catching Spinning Table Tennis Balls in Simulation with End-to-End Curriculum Reinforcement Learning](https://mlsanity.com/paper/2503.01251/) (2025)
- [SpikePingpong: Spike Vision-based Fast-Slow Pingpong Robot System](https://mlsanity.com/paper/2506.06690/) (2025)

## Abstract (arXiv)

We propose a model-free algorithm for learning efficient policies capable of returning table tennis balls by controlling robot joints at a rate of 100Hz. We demonstrate that evolutionary search (ES) methods acting on CNN-based policy architectures for non-visual inputs and convolving across time learn compact controllers leading to smooth motions. Furthermore, we show that with appropriately tuned curriculum learning on the task and rewards, policies are capable of developing multi-modal styles, specifically forehand and backhand stroke, whilst achieving 80% return rate on a wide range of ball throws. We observe that multi-modality does not require any architectural priors, such as multi-head architectures or hierarchical policies.