# Learning Bipedal Walking for Humanoids with Current Feedback

arXiv: https://arxiv.org/abs/2303.03724 · HTML: https://mlsanity.com/paper/2303.03724/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 37 references that are in the map.
- First posted on arXiv 7 Mar 2023.

## Builds on (most cited first, 22 of 22)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Sim-to-Real: Learning Agile Locomotion For Quadruped Robots](https://mlsanity.com/paper/1804.10332/) (2018)
- [Learning Agile Robotic Locomotion Skills by Imitating Animals](https://mlsanity.com/paper/2004.00784/) (2020)
- [Real-World Humanoid Locomotion with Reinforcement Learning](https://mlsanity.com/paper/2303.03381/) (2023)
- [Rapid Locomotion via Reinforcement Learning](https://mlsanity.com/paper/2205.02824/) (2022)
- [Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Composition](https://mlsanity.com/paper/2011.01387/) (2020)
- [Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learning](https://mlsanity.com/paper/2105.08328/) (2021)
- [Multi-expert learning of adaptive legged locomotion](https://mlsanity.com/paper/2012.05810/) (2020)
- [Sim-to-Real Transfer for Biped Locomotion](https://mlsanity.com/paper/1903.01390/) (2019)
- [Adapting Rapid Motor Adaptation for Bipedal Robots](https://mlsanity.com/paper/2205.15299/) (2022)
- [Learning Memory-Based Control for Human-Scale Bipedal Locomotion](https://mlsanity.com/paper/2006.02402/) (2020)
- [DeepWalk: Omnidirectional Bipedal Gait by Deep Reinforcement Learning](https://mlsanity.com/paper/2106.00534/) (2021)
- [Learning Bipedal Walking On Planned Footsteps For Humanoid Robots](https://mlsanity.com/paper/2207.12644/) (2022)
- [Stair Climbing Stabilization of the HRP-4 Humanoid Robot using Whole-body Admittance Control](https://mlsanity.com/paper/1809.07073/) (2018)
- [Online Non-linear Centroidal MPC for Humanoid Robot Locomotion with Step Adjustment](https://mlsanity.com/paper/2203.04489/) (2022)
- [Learning Whole-body Motor Skills for Humanoids](https://mlsanity.com/paper/2002.02991/) (2020)
- [mc-mujoco: Simulating Articulated Robots with FSM Controllers in MuJoCo](https://mlsanity.com/paper/2209.00274/) (2022)
- [Sim-to-Real Transfer of Compliant Bipedal Locomotion on Torque Sensor-Less Gear-Driven Humanoid](https://mlsanity.com/paper/2204.03897/) (2022)
- [Learning Linear Policies for Robust Bipedal Locomotion on Terrains with Varying Slopes](https://mlsanity.com/paper/2104.01662/) (2021)

## Built on by (most cited first, 18 of 18)

- [HumanPlus: Humanoid Shadowing and Imitation from Humans](https://mlsanity.com/paper/2406.10454/) (2024)
- [Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning](https://mlsanity.com/paper/2304.13653/) (2023)
- [Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control](https://mlsanity.com/paper/2401.16889/) (2024)
- [PDP: Physics-Based Character Animation via Diffusion Policy](https://mlsanity.com/paper/2406.00960/) (2024)
- [Whole-body Humanoid Robot Locomotion with Human Reference](https://mlsanity.com/paper/2402.18294/) (2024)
- [Distillation-PPO: A Novel Two-Stage Reinforcement Learning Framework for Humanoid Robot Perceptive Locomotion](https://mlsanity.com/paper/2503.08299/) (2025)
- [Deep Reinforcement Learning for Bipedal Locomotion: A Brief Survey](https://mlsanity.com/paper/2404.17070/) (2024)
- [Enhancing Model-Based Step Adaptation for Push Recovery through Reinforcement Learning of Step Timing and Region](https://mlsanity.com/paper/2411.01000/) (2024)
- [Robust Humanoid Walking on Compliant and Uneven Terrain with Deep Reinforcement Learning](https://mlsanity.com/paper/2504.13619/) (2025)
- [Learning Bipedal Locomotion on Gear-Driven Humanoid Robot Using Foot-Mounted IMUs](https://mlsanity.com/paper/2504.00614/) (2025)
- [Efficiently Learning Robust Torque-based Locomotion Through Reinforcement with Model-Based Supervision](https://mlsanity.com/paper/2601.16109/) (2026)
- [Training and Evaluating Diffusion Policies with Long Context Lengths](https://mlsanity.com/paper/2606.16447/) (2026)
- [Learning Skateboarding for Humanoid Robots through Massively Parallel Reinforcement Learning](https://mlsanity.com/paper/2409.07846/) (2024)
- [Learning to enhance multi-legged robot on rugged landscapes](https://mlsanity.com/paper/2409.09473/) (2024)
- [Achieving Precise and Reliable Locomotion with Differentiable Simulation-Based System Identification](https://mlsanity.com/paper/2508.04696/) (2025)
- [ECO: Energy-Constrained Optimization with Reinforcement Learning for Humanoid Walking](https://mlsanity.com/paper/2602.06445/) (2026)
- [HuMam: Humanoid Motion Control via End-to-End Deep Reinforcement Learning with Mamba](https://mlsanity.com/paper/2509.18046/) (2025)
- [GaitSpan: Growing Humanoid Locomotion from Walking to Running](https://mlsanity.com/paper/2607.12114/) (2026)

## Abstract (arXiv)

Recent advances in deep reinforcement learning (RL) based techniques combined with training in simulation have offered a new approach to developing robust controllers for legged robots. However, the application of such approaches to real hardware has largely been limited to quadrupedal robots with direct-drive actuators and light-weight bipedal robots with low gear-ratio transmission systems. Application to real, life-sized humanoid robots has been less common arguably due to a large sim2real gap. In this paper, we present an approach for effectively overcoming the sim2real gap issue for humanoid robots arising from inaccurate torque-tracking at the actuator level. Our key idea is to utilize the current feedback from the actuators on the real robot, after training the policy in a simulation environment artificially degraded with poor torque-tracking. Our approach successfully trains a unified, end-to-end policy in simulation that can be deployed on a real HRP-5P humanoid robot to achieve bipedal locomotion. Through ablations, we also show that a feedforward policy architecture combined with targeted dynamics randomization is sufficient for zero-shot sim2real success, thus eliminating the need for computationally expensive, memory-based network architectures. Finally, we validate the robustness of the proposed RL policy by comparing its performance against a conventional model-based controller for walking on uneven terrain with the real robot.