# DeepWalk: Omnidirectional Bipedal Gait by Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2106.00534 · HTML: https://mlsanity.com/paper/2106.00534/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jun 2021.

## Built on by (most cited first, 21 of 21)

- [Real-World Humanoid Locomotion with Reinforcement Learning](https://mlsanity.com/paper/2303.03381/) (2023)
- [Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control](https://mlsanity.com/paper/2401.16889/) (2024)
- [GenLoco: Generalized Locomotion Controllers for Quadrupedal Robots](https://mlsanity.com/paper/2209.05309/) (2022)
- [Robust and Versatile Bipedal Jumping Control through Reinforcement Learning](https://mlsanity.com/paper/2302.09450/) (2023)
- [Learning Humanoid Locomotion over Challenging Terrain](https://mlsanity.com/paper/2410.03654/) (2024)
- [CurricuLLM: Automatic Task Curricula Design for Learning Complex Robot Skills using Large Language Models](https://mlsanity.com/paper/2409.18382/) (2024)
- [Learning Bipedal Walking for Humanoids with Current Feedback](https://mlsanity.com/paper/2303.03724/) (2023)
- [Learning Bipedal Walking On Planned Footsteps For Humanoid Robots](https://mlsanity.com/paper/2207.12644/) (2022)
- [Residual Physics Learning and System Identification for Sim-to-real Transfer of Policies on Buoyancy Assisted Legged Robots](https://mlsanity.com/paper/2303.09597/) (2023)
- [Deep Reinforcement Learning for Bipedal Locomotion: A Brief Survey](https://mlsanity.com/paper/2404.17070/) (2024)
- [Achieving Stable High-Speed Locomotion for Humanoid Robots with Deep Reinforcement Learning](https://mlsanity.com/paper/2409.16611/) (2024)
- [Reinforcement Learning with Adaptive Curriculum Dynamics Randomization for Fault-Tolerant Robot Control](https://mlsanity.com/paper/2111.10005/) (2021)
- [Robust Humanoid Walking on Compliant and Uneven Terrain with Deep Reinforcement Learning](https://mlsanity.com/paper/2504.13619/) (2025)
- [Contrastive Representation Learning for Robust Sim-to-Real Transfer of Adaptive Humanoid Locomotion](https://mlsanity.com/paper/2509.12858/) (2025)
- [Machine Learning Meets Advanced Robotic Manipulation](https://mlsanity.com/paper/2309.12560/) (2023)
- [Learning Skateboarding for Humanoid Robots through Massively Parallel Reinforcement Learning](https://mlsanity.com/paper/2409.07846/) (2024)
- [Learning Bipedal Walking for Humanoid Robots in Challenging Environments with Obstacle Avoidance](https://mlsanity.com/paper/2410.08212/) (2024)
- [Sim-to-Real Transfer in Deep Reinforcement Learning for Bipedal Locomotion](https://mlsanity.com/paper/2511.06465/) (2025)
- [Custom Sine Waves Are Enough for Imitation Learning of Bipedal Gaits with Different Styles](https://mlsanity.com/paper/2204.04157/) (2022)
- [Integrating Controllable Motion Skills from Demonstrations](https://mlsanity.com/paper/2408.03018/) (2024)
- [HuMam: Humanoid Motion Control via End-to-End Deep Reinforcement Learning with Mamba](https://mlsanity.com/paper/2509.18046/) (2025)

## Abstract (arXiv)

Bipedal walking is one of the most difficult but exciting challenges in robotics. The difficulties arise from the complexity of high-dimensional dynamics, sensing and actuation limitations combined with real-time and computational constraints. Deep Reinforcement Learning (DRL) holds the promise to address these issues by fully exploiting the robot dynamics with minimal craftsmanship. In this paper, we propose a novel DRL approach that enables an agent to learn omnidirectional locomotion for humanoid (bipedal) robots. Notably, the locomotion behaviors are accomplished by a single control policy (a single neural network). We achieve this by introducing a new curriculum learning method that gradually increases the task difficulty by scheduling target velocities. In addition, our method does not require reference motions which facilities its application to robots with different kinematics, and reduces the overall complexity. Finally, different strategies for sim-to-real transfer are presented which allow us to transfer the learned policy to a real humanoid robot.