# Sim-to-Real Learning for Bipedal Locomotion Under Unsensed Dynamic Loads

arXiv: https://arxiv.org/abs/2204.04340 · HTML: https://mlsanity.com/paper/2204.04340/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 25 references that are in the map.
- First posted on arXiv 9 Apr 2022.

## Builds on (most cited first, 12 of 12)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Sim-to-Real: Learning Agile Locomotion For Quadruped Robots](https://mlsanity.com/paper/1804.10332/) (2018)
- [DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills](https://mlsanity.com/paper/1804.02717/) (2018)
- [Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Composition](https://mlsanity.com/paper/2011.01387/) (2020)
- [Sim-to-Real Transfer for Biped Locomotion](https://mlsanity.com/paper/1903.01390/) (2019)
- [A Unified MPC Framework for Whole-Body Dynamic Locomotion and Manipulation](https://mlsanity.com/paper/2103.00946/) (2021)
- [AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control](https://mlsanity.com/paper/2104.02180/) (2021)
- [Learning natural locomotion behaviors for humanoid robots using human knowledge](https://mlsanity.com/paper/2005.10195/) (2020)
- [Learning Bipedal Robot Locomotion from Human Movement](https://mlsanity.com/paper/2105.12277/) (2021)
- [Optimisation of Body-ground Contact for Augmenting Whole-Body Loco-manipulation of Quadruped Robots](https://mlsanity.com/paper/2002.10552/) (2020)
- [ResQbot: A Mobile Rescue Robot with Immersive Teleperception for Casualty Extraction](https://mlsanity.com/paper/1812.09134/) (2018)

## Built on by (most cited first, 10 of 10)

- [OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning](https://mlsanity.com/paper/2406.08858/) (2024)
- [Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control](https://mlsanity.com/paper/2401.16889/) (2024)
- [DiffuseLoco: Real-Time Legged Locomotion Control with Diffusion from Offline Datasets](https://mlsanity.com/paper/2404.19264/) (2024)
- [A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Models](https://mlsanity.com/paper/2502.13187/) (2025)
- [Learning Multi-Modal Whole-Body Control for Real-World Humanoid Robots](https://mlsanity.com/paper/2408.07295/) (2024)
- [Infer and Adapt: Bipedal Locomotion Reward Learning from Demonstrations via Inverse Reinforcement Learning](https://mlsanity.com/paper/2309.16074/) (2023)
- [Optimizing Bipedal Locomotion for The 100m Dash With Comparison to Human Running](https://mlsanity.com/paper/2508.03070/) (2025)
- [Learning Decentralized Multi-Biped Control for Payload Transport](https://mlsanity.com/paper/2406.17279/) (2024)
- [From Abstraction to Reality: DARPA's Vision for Robust Sim-to-Real Autonomy](https://mlsanity.com/paper/2503.11007/) (2025)
- [Enabling Efficient, Reliable Real-World Reinforcement Learning with Approximate Physics-Based Models](https://mlsanity.com/paper/2307.08168/) (2023)

## Abstract (arXiv)

Recent work on sim-to-real learning for bipedal locomotion has demonstrated new levels of robustness and agility over a variety of terrains. However, that work, and most prior bipedal locomotion work, have not considered locomotion under a variety of external loads that can significantly influence the overall system dynamics. In many applications, robots will need to maintain robust locomotion under a wide range of potential dynamic loads, such as pulling a cart or carrying a large container of sloshing liquid, ideally without requiring additional load-sensing capabilities. In this work, we explore the capabilities of reinforcement learning (RL) and sim-to-real transfer for bipedal locomotion under dynamic loads using only proprioceptive feedback. We show that prior RL policies trained for unloaded locomotion fail for some loads and that simply training in the context of loads is enough to result in successful and improved policies. We also compare training specialized policies for each load versus a single policy for all considered loads and analyze how the resulting gaits change to accommodate different loads. Finally, we demonstrate sim-to-real transfer, which is successful but shows a wider sim-to-real gap than prior unloaded work, which points to interesting future research.