# PACE: Physics Augmentation for Coordinated End-to-end Reinforcement Learning toward Versatile Humanoid Table Tennis

arXiv: https://arxiv.org/abs/2509.21690 · HTML: https://mlsanity.com/paper/2509.21690/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 20 of its 37 references that are in the map.
- First posted on arXiv 25 Sep 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 20 of 20)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments](https://mlsanity.com/paper/2301.04195/) (2023)
- [Asymmetric Actor Critic for Image-Based Robot Learning](https://mlsanity.com/paper/1710.06542/) (2017)
- [Real-World Humanoid Locomotion with Reinforcement Learning](https://mlsanity.com/paper/2303.03381/) (2023)
- [ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills](https://mlsanity.com/paper/2502.01143/) (2025)
- [ExBody2: Advanced Expressive Humanoid Whole-Body Control](https://mlsanity.com/paper/2412.13196/) (2024)
- [Learning to Play Table Tennis From Scratch using Muscular Robots](https://mlsanity.com/paper/2006.05935/) (2020)
- [HITTER: A HumanoId Table TEnnis Robot via Hierarchical Planning and Learning](https://mlsanity.com/paper/2508.21043/) (2025)
- [Achieving Human Level Competitive Robot Table Tennis](https://mlsanity.com/paper/2408.03906/) (2024)
- [Strategy and Skill Learning for Physics-based Table Tennis Animation](https://mlsanity.com/paper/2407.16210/) (2024)
- [Robotic Table Tennis with Model-Free Reinforcement Learning](https://mlsanity.com/paper/2003.14398/) (2020)
- [Sample-efficient Reinforcement Learning in Robotic Table Tennis](https://mlsanity.com/paper/2011.03275/) (2020)
- [GoalsEye: Learning High Speed Precision Table Tennis on a Physical Robot](https://mlsanity.com/paper/2210.03662/) (2022)
- [Safe & Accurate at Speed with Tendons: A Robot Arm for Exploring Dynamic Motion](https://mlsanity.com/paper/2307.02654/) (2023)
- [avaTTAR: Table Tennis Stroke Training with On-body and Detached Visualization in Augmented Reality](https://mlsanity.com/paper/2407.15373/) (2024)
- [Booster Gym: An End-to-End Reinforcement Learning Framework for Humanoid Robot Locomotion](https://mlsanity.com/paper/2506.15132/) (2025)
- [Hindsight States: Blending Sim and Real Task Elements for Efficient Reinforcement Learning](https://mlsanity.com/paper/2303.02234/) (2023)
- [Adaptive Ankle Torque Control for Bipedal Humanoid Walking on Surfaces with Unknown Horizontal and Vertical Motion](https://mlsanity.com/paper/2410.11799/) (2024)
- [High Speed Robotic Table Tennis Swinging Using Lightweight Hardware with Model Predictive Control](https://mlsanity.com/paper/2505.01617/) (2025)

## Built on by (most cited first, 2 of 2)

- [Humanoid Goalkeeper: Learning from Position Conditioned Task-Motion Constraints](https://mlsanity.com/paper/2510.18002/) (2025)
- [RobotWorld: Benchmarking Multimodal Agents for Robot Use Across Diverse Tasks and Embodiments](https://mlsanity.com/paper/2610.10409/) (2026)

## Abstract (arXiv)

Humanoid table tennis (TT) demands rapid perception, proactive whole-body motion, and agile footwork under strict timing--capabilities that remain difficult for end-to-end control policies. We propose a reinforcement learning (RL) framework that maps ball-position observations directly to whole-body joint commands for both arm striking and leg locomotion, strengthened by predictive signals and dense, physics-guided rewards. A lightweight learned predictor, fed with recent ball positions, estimates future ball states and augments the policy's observations for proactive decision-making. During training, a physics-based predictor supplies precise future states to construct dense, informative rewards that lead to effective exploration. The resulting policy attains strong performance across varied serve ranges (hit rate≥96% and success rate≥92%) in simulations. Ablation studies confirm that both the learned predictor and the predictive reward design are critical for end-to-end learning. Deployed zero-shot on a physical Booster T1 humanoid with 23 revolute joints, the policy produces coordinated lateral and forward-backward footwork with accurate, fast returns, suggesting a practical path toward versatile, competitive humanoid TT. We have open-sourced our RL training code at: https://github.com/purdue-tracelab/TTRL-ICRA2026