# Bridging the Sim-to-Real Gap for Athletic Loco-Manipulation

arXiv: https://arxiv.org/abs/2502.10894 · HTML: https://mlsanity.com/paper/2502.10894/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 52 references that are in the map.
- First posted on arXiv 15 Feb 2025.

## Builds on (most cited first, 30 of 35)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Sim-to-Real: Learning Agile Locomotion For Quadruped Robots](https://mlsanity.com/paper/1804.10332/) (2018)
- [Perpetual Humanoid Control for Real-time Simulated Avatars](https://mlsanity.com/paper/2305.06456/) (2023)
- [Real-World Humanoid Locomotion with Reinforcement Learning](https://mlsanity.com/paper/2303.03381/) (2023)
- [Expressive Whole-Body Control for Humanoid Robots](https://mlsanity.com/paper/2402.16796/) (2024)
- [TossingBot: Learning to Throw Arbitrary Objects with Residual Physics](https://mlsanity.com/paper/1903.11239/) (2019)
- [Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning](https://mlsanity.com/paper/2304.13653/) (2023)
- [Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation](https://mlsanity.com/paper/2403.04436/) (2024)
- [Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotion](https://mlsanity.com/paper/2210.10044/) (2022)
- [Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control](https://mlsanity.com/paper/2401.16889/) (2024)
- [Concurrent Training of a Control Policy and a State Estimator for Dynamic and Robust Legged Locomotion](https://mlsanity.com/paper/2202.05481/) (2022)
- [UMI on Legs: Making Manipulation Policies Mobile with Manipulation-Centric Whole-body Controllers](https://mlsanity.com/paper/2407.10353/) (2024)
- [DreamWaQ: Learning Robust Quadrupedal Locomotion With Implicit Terrain Imagination via Deep Reinforcement Learning](https://mlsanity.com/paper/2301.10602/) (2023)
- [Visual Whole-Body Control for Legged Loco-Manipulation](https://mlsanity.com/paper/2403.16967/) (2024)
- [WoCoCo: Learning Whole-Body Humanoid Control with Sequential Contacts](https://mlsanity.com/paper/2406.06005/) (2024)
- [Dynamics Randomization Revisited:A Case Study for Quadrupedal Locomotion](https://mlsanity.com/paper/2011.02404/) (2020)
- [DribbleBot: Dynamic Legged Manipulation in the Wild](https://mlsanity.com/paper/2304.01159/) (2023)
- [Hybrid Internal Model: Learning Agile Legged Locomotion with Simulated Robot Response](https://mlsanity.com/paper/2312.11460/) (2023)
- [A Unified MPC Framework for Whole-Body Dynamic Locomotion and Manipulation](https://mlsanity.com/paper/2103.00946/) (2021)
- [Hierarchical Reinforcement Learning for Precise Soccer Shooting Skills using a Quadrupedal Robot](https://mlsanity.com/paper/2208.01160/) (2022)
- [Combining Learning-based Locomotion Policy with Model-based Manipulation for Legged Mobile Manipulators](https://mlsanity.com/paper/2201.03871/) (2022)
- [SMPLOlympics: Sports Environments for Physically Simulated Humanoids](https://mlsanity.com/paper/2407.00187/) (2024)
- [Learning Humanoid Locomotion over Challenging Terrain](https://mlsanity.com/paper/2410.03654/) (2024)
- [Learning Force Control for Legged Manipulation](https://mlsanity.com/paper/2405.01402/) (2024)

## Built on by (most cited first, 13 of 13)

- [Empirical Analysis of Sim-and-Real Cotraining of Diffusion Policies for Planar Pushing from Pixels](https://mlsanity.com/paper/2503.22634/) (2025)
- [FACET: Force-Adaptive Control via Impedance Reference Tracking for Legged Robots](https://mlsanity.com/paper/2505.06883/) (2025)
- [DexNDM: Closing the Reality Gap for Dexterous In-Hand Rotation via Joint-Wise Neural Dynamics Model](https://mlsanity.com/paper/2510.08556/) (2025)
- [GentleHumanoid: Learning Upper-body Compliance for Contact-rich Human and Object Interaction](https://mlsanity.com/paper/2511.04679/) (2025)
- [Closing the Reality Gap: Zero-Shot Sim-to-Real Deployment for Dexterous Force-Based Grasping and Manipulation](https://mlsanity.com/paper/2601.02778/) (2026)
- [Constrained Reinforcement Learning for Unstable Point-Feet Bipedal Locomotion Applied to the Bolt Robot](https://mlsanity.com/paper/2508.02194/) (2025)
- [HERO: Learning Humanoid End-Effector Control for Visual Whole-Body Open-Vocabulary Object Grasping](https://mlsanity.com/paper/2602.16705/) (2026)
- [HALO:Closing Sim-to-Real Gap for Heavy-loaded Humanoid Agile Motion Skills via Differentiable Simulation](https://mlsanity.com/paper/2603.15084/) (2026)
- [NeuralActuator: Neural Actuation Modeling for Robot Dynamics and External Force Perception](https://mlsanity.com/paper/2607.11734/) (2026)
- [An Real-Sim-Real (RSR) Loop Framework for Generalizable Robotic Policy Transfer](https://mlsanity.com/paper/2503.10118/) (2025)
- [Sim-to-Real Transfer for Muscle-Actuated Robots via Generalized Actuator Networks](https://mlsanity.com/paper/2604.09487/) (2026)
- [Learning Dynamic Pick-and-Place for a Legged Manipulator](https://mlsanity.com/paper/2605.15713/) (2026)
- [TAM: Torque Adaptation Module for Robust Motion Transfer in Manipulation](https://mlsanity.com/paper/2606.06218/) (2026)

## Abstract (arXiv)

Achieving athletic loco-manipulation on robots requires moving beyond traditional tracking rewards - which simply guide the robot along a reference trajectory - to task rewards that drive truly dynamic, goal-oriented behaviors. Commands such as "throw the ball as far as you can" or "lift the weight as quickly as possible" compel the robot to exhibit the agility and power inherent in athletic performance. However, training solely with task rewards introduces two major challenges: these rewards are prone to exploitation (reward hacking), and the exploration process can lack sufficient direction. To address these issues, we propose a two-stage training pipeline. First, we introduce the Unsupervised Actuator Net (UAN), which leverages real-world data to bridge the sim-to-real gap for complex actuation mechanisms without requiring access to torque sensing. UAN mitigates reward hacking by ensuring that the learned behaviors remain robust and transferable. Second, we use a pre-training and fine-tuning strategy that leverages reference trajectories as initial hints to guide exploration. With these innovations, our robot athlete learns to lift, throw, and drag with remarkable fidelity from simulation to reality.