mlsanityData to 14 Oct 2026

Sim-to-Real Transfer in Deep Reinforcement Learning for Bipedal Locomotion

arXiv 2511.06465 ↗First posted 9 Nov 2025cs.ROcited by 1

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

This chapter addresses the critical challenge of simulation-to-reality (sim-to-real) transfer for deep reinforcement learning (DRL) in bipedal locomotion. After contextualizing the problem within various control architectures, we dissect the ``curse of simulation'' by analyzing the primary sources of sim-to-real gap: robot dynamics, contact modeling, state estimation, and numerical solvers. Building on this diagnosis, we structure the solutions around two complementary philosophies. The first is to shrink the gap through model-centric strategies that systematically improve the simulator's physical fidelity. The second is to harden the policy, a complementary approach that uses in-simulation robustness training and post-deployment adaptation to make the policy inherently resilient to model inaccuracies. The chapter concludes by synthesizing these philosophies into a strategic framework, providing a clear roadmap for developing and evaluating robust sim-to-real solutions.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 24 of its 30 references that are in the map. First posted on arXiv 9 Nov 2025.

Builds on

24 of 30 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control30 Jan

Other references · 23

  1. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  2. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  3. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
  4. 2019Learning agile and dynamic motor skills for legged robotscited by 504
  5. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 319
  6. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 213
  7. 2023Real-World Humanoid Locomotion with Reinforcement Learningcited by 187
  8. 2024HumanPlus: Humanoid Shadowing and Imitation from Humanscited by 185
  9. 2025ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skillscited by 130
  10. 2021Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robotscited by 116
  11. 2020Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Compositioncited by 96
  12. 2018Learning Symmetric and Low-energy Locomotioncited by 61
11 more
  1. 2019Sim-to-Real Transfer for Biped Locomotioncited by 56
  2. 2020ALLSTEPS: Curriculum-driven Learning of Stepping Stone Skillscited by 54
  3. 2022Adapting Rapid Motor Adaptation for Bipedal Robotscited by 54
  4. 2016Learning Locomotion Skills Using DeepRL: Does the Choice of Action Space Matter?cited by 48
  5. 2020Learning Memory-Based Control for Human-Scale Bipedal Locomotioncited by 28
  6. 2020Learning Task Space Actions for Bipedal Locomotioncited by 21
  7. 2021DeepWalk: Omnidirectional Bipedal Gait by Deep Reinforcement Learningcited by 21
  8. 2020Learning Spring Mass Locomotion: Guiding Policies with a Reduced-Order Modelcited by 15
  9. 2022Learning Bipedal Walking On Planned Footsteps For Humanoid Robotscited by 12
  10. 2023Residual Physics Learning and System Identification for Sim-to-real Transfer of Policies on Buoyancy Assisted Legged Robotscited by 8
  11. 2024Physically Consistent Online Inertial Adaptation for Humanoid Loco-manipulationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Application of Deep Reinforcement Learning to Event-Triggered Control for Networked Artificial Pancreas Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey
  2. 2024Deep Reinforcement Learning for Bipedal Locomotion: A Brief Survey
  3. 2025An Real-Sim-Real (RSR) Loop Framework for Generalizable Robotic Policy Transfer
  4. 2025Towards bridging the gap: Systematic sim-to-real transfer for diverse legged robots
  5. 2025Sim-to-Real Transfer for Mobile Robots with Reinforcement Learning: from NVIDIA Isaac Sim to Gazebo and Real ROS 2 Robots
  6. 2025Robust Humanoid Walking on Compliant and Uneven Terrain with Deep Reinforcement Learning
  7. 2025Revealing the Challenges of Sim-to-Real Transfer in Model-Based Reinforcement Learning via Latent Space Modeling
  8. 2021Adaptive Mimic: Deep Reinforcement Learning of Parameterized Bipedal Walking from Infeasible References ↗