mlsanityData to 14 Oct 2026

Learning Sim-to-Real Humanoid Locomotion in 15 Minutes

arXiv 2512.01996 ↗First posted 1 Dec 2025cs.RO cs.AI cs.LGcited by 16

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Massively parallel simulation has reduced reinforcement learning (RL) training time for robots from days to minutes. However, achieving fast and reliable sim-to-real RL for humanoid control remains difficult due to the challenges introduced by factors such as high dimensionality and domain randomization. In this work, we introduce a simple and practical recipe based on off-policy RL algorithms, i.e., FastSAC and FastTD3, that enables rapid training of humanoid locomotion policies in just 15 minutes with a single RTX 4090 GPU. Our simple recipe stabilizes off-policy RL algorithms at massive scale with thousands of parallel environments through carefully tuned design choices and minimalist reward functions. We demonstrate rapid end-to-end learning of humanoid locomotion controllers on Unitree G1 and Booster T1 robots under strong domain randomization, e.g., randomized dynamics, rough terrain, and push perturbations, as well as fast training of whole-body human-motion tracking policies. We provide videos and open-source implementation at: https://younggyo.me/fastsac-humanoid.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 41 of its 51 references that are in the map. First posted on arXiv 1 Dec 2025.

Builds on

41 of 51 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025RSL-RL: A Learning Library for Robotics Research13 Sep
  2. 2025FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid Control28 May
  3. 2024Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control30 Jan

Other references · 38

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2016Layer Normalizationcited by 5,669
  3. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  5. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  6. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  7. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  8. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  9. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  10. 2019Solving Rubik's Cube with a Robot Handcited by 734
  11. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  12. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
26 more
  1. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 474
  2. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
  3. 2023Efficient Online Reinforcement Learning with Offline Datacited by 363
  4. 2023Orbit: A Unified Simulation Framework for Interactive Robot Learning Environmentscited by 295
  5. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 233
  6. 2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 217
  7. 2017Emergence of Locomotion Behaviours in Rich Environmentscited by 165
  8. 2025BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusioncited by 165
  9. 2023Extreme Parkour with Legged Robotscited by 164
  10. 2018Learning to Walk via Deep Reinforcement Learningcited by 158
  11. 2022Legged Locomotion in Challenging Terrains using Egocentric Visioncited by 151
  12. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 147
  13. 2025ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skillscited by 130
  14. 2024Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous controlcited by 108
  15. 2024Humanoid Parkour Learningcited by 96
  16. 2024HOVER: Versatile Neural Whole-Body Controller for Humanoid Robotscited by 91
  17. 2018GPU-Accelerated Robotic Simulation for Distributed Reinforcement Learningcited by 90
  18. 2025MuJoCo Playgroundcited by 78
  19. 2024SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learningcited by 76
  20. 2024DextrAH-RGB: Visuomotor Policies to Grasp Anything with Dexterous Handscited by 45
  21. 2018Accelerated Methods for Deep Reinforcement Learningcited by 40
  22. 2024MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximizationcited by 30
  23. 2022Learning Free Gait Transition for Quadruped Robots via Phase-Guided Controllercited by 29
  24. 2024Symmetry Considerations for Learning Task Symmetric Robot Policiescited by 28
  25. 2024Learning Humanoid Locomotion over Challenging Terraincited by 26
  26. 2025Simplicial Embeddings Improve Sample Efficiency in Actor-Critic Agentscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 16.

16 papers.

Sort:

By area:

  1. 2026FlashSAC: Fast and Stable Off-Policy Reinforcement Learning for High-Dimensional Robot Controlcited by 5
  2. 2026CLOT: Closed-Loop Global Motion Tracking for Whole-Body Humanoid Teleoperationcited by 4
  3. 2026Squint: Fast Visual Reinforcement Learning for Sim-to-Real Roboticscited by 2
  4. 2026Trust-Region Diffusion Policies for Massively Parallel On-Policy RLcited by 2
  5. 2026FastDSAC: Unlocking the Potential of Maximum Entropy RL in High-Dimensional Humanoid Controlcited by 1
  6. 2026Dream-MPC: Gradient-Based Model Predictive Control with Latent Imaginationcited by 1
  7. 2026Self-Evolving AI for Humanoids: Mechanisms, Safety, and Evaluation of Post-Deployment Self-Improvementcited by 1
  8. 2026Sim-to-Real Transfer for Muscle-Actuated Robots via Generalized Actuator Networkscited by 0
  9. 2026Constraint-Enhanced Reinforcement Learning Based on Dynamic Decoupled Spherical Radial Squashingcited by 0
  10. 2026When Does Non-Uniform Replay Matter in Reinforcement Learning?cited by 0
  11. 2026Bridging the Gap: Enabling Soft Actor Critic for High Performance Legged Locomotioncited by 0
  12. 2026FlashNav: Training Deployable Robot Navigation Policies in Secondscited by 0
4 more
  1. 2026FastDSAC: Enhancing Policy Plasticity via Constrained Exploration for Scalable Humanoid Locomotioncited by 0
  2. 2026Rethinking the Suitability of Reinforcement Learning Algorithms Under Practical Transfer Constraintscited by 0
  3. 2026Locomotion-Grounded Humanoid Soccer: Task-Gated Reinforcement Learning of a Multi-Directional Kicking Librarycited by 0
  4. 2026ACCRUE: Continual Off-Policy Reinforcement Learning for Robot Foundation Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Towards Bridging the Gap between Large-Scale Pretraining and Efficient Finetuning for Humanoid Control
  2. 2025Robot Trains Robot: Automatic Real-World Policy Adaptation and Learning for Humanoids
  3. 2025LiPS: Large-Scale Humanoid Robot Reinforcement Learning with Parallel-Series Structures
  4. 2025Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfer
  5. 2026Accelerating and Scaling MPC-Guided Reinforcement Learning for Humanoid Locomotion and Manipulation
  6. 2025Booster Gym: An End-to-End Reinforcement Learning Framework for Humanoid Robot Locomotion
  7. 2025Robust Humanoid Walking on Compliant and Uneven Terrain with Deep Reinforcement Learning
  8. 2025Mechanical Intelligence-Aware Curriculum Reinforcement Learning for Humanoids with Parallel Actuation