# Learning Humanoid Locomotion over Challenging Terrain

arXiv: https://arxiv.org/abs/2410.03654 · HTML: https://mlsanity.com/paper/2410.03654/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 55 references that are in the map.
- First posted on arXiv 4 Oct 2024.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [The KIT Motion-Language Dataset](https://mlsanity.com/paper/1607.03827/) (2016)
- [Asymmetric Actor Critic for Image-Based Robot Learning](https://mlsanity.com/paper/1710.06542/) (2017)
- [Real-World Humanoid Locomotion with Reinforcement Learning](https://mlsanity.com/paper/2303.03381/) (2023)
- [PoseTrack: A Benchmark for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1710.10000/) (2017)
- [Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Composition](https://mlsanity.com/paper/2011.01387/) (2020)
- [Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learning](https://mlsanity.com/paper/2105.08328/) (2021)
- [Humanoid Locomotion as Next Token Prediction](https://mlsanity.com/paper/2402.19469/) (2024)
- [EnvPool: A Highly Parallel Reinforcement Learning Environment Execution Engine](https://mlsanity.com/paper/2206.10558/) (2022)
- [DeepWalk: Omnidirectional Bipedal Gait by Deep Reinforcement Learning](https://mlsanity.com/paper/2106.00534/) (2021)
- [Feedback Control of a Cassie Bipedal Robot: Walking, Standing, and Riding a Segway](https://mlsanity.com/paper/1809.07279/) (2018)
- [Robust Feedback Motion Policy Design Using Reinforcement Learning on a 3D Digit Bipedal Robot](https://mlsanity.com/paper/2103.15309/) (2021)
- [Dynamic Locomotion For Passive-Ankle Biped Robots And Humanoids Using Whole-Body Locomotion Control](https://mlsanity.com/paper/1901.08100/) (2019)
- [Footstep Planning for Autonomous Walking Over Rough Terrain](https://mlsanity.com/paper/1907.08673/) (2019)

## Built on by (most cited first, 26 of 26)

- [TWIST: Teleoperated Whole-Body Imitation System](https://mlsanity.com/paper/2505.02833/) (2025)
- [Visual Imitation Enables Contextual Humanoid Control](https://mlsanity.com/paper/2505.03729/) (2025)
- [BeamDojo: Learning Agile Humanoid Locomotion on Sparse Footholds](https://mlsanity.com/paper/2502.10363/) (2025)
- [Learning Smooth Humanoid Locomotion through Lipschitz-Constrained Policies](https://mlsanity.com/paper/2410.11825/) (2024)
- [Whole-Body Conditioned Egocentric Video Prediction](https://mlsanity.com/paper/2506.21552/) (2025)
- [Learning from Massive Human Videos for Universal Humanoid Pose Control](https://mlsanity.com/paper/2412.14172/) (2024)
- [Learning Getting-Up Policies for Real-World Humanoid Robots](https://mlsanity.com/paper/2502.12152/) (2025)
- [Learning Sim-to-Real Humanoid Locomotion in 15 Minutes](https://mlsanity.com/paper/2512.01996/) (2025)
- [Bridging the Sim-to-Real Gap for Athletic Loco-Manipulation](https://mlsanity.com/paper/2502.10894/) (2025)
- [Learning Humanoid Locomotion with World Model Reconstruction](https://mlsanity.com/paper/2502.16230/) (2025)
- [Hiking in the Wild: A Scalable Perceptive Parkour Framework for Humanoids](https://mlsanity.com/paper/2601.07718/) (2026)
- [Why AI systems don't learn and what to do about it: Lessons on autonomous learning from cognitive science](https://mlsanity.com/paper/2603.15381/) (2026)
- [LookOut: Real-World Humanoid Egocentric Navigation](https://mlsanity.com/paper/2508.14466/) (2025)
- [Robot Trains Robot: Automatic Real-World Policy Adaptation and Learning for Humanoids](https://mlsanity.com/paper/2508.12252/) (2025)
- [From Generated Human Videos to Physically Plausible Robot Trajectories](https://mlsanity.com/paper/2512.05094/) (2025)
- [DreamPolicy: A Unified World-model Policy for Scalable Humanoid Locomotion](https://mlsanity.com/paper/2505.18780/) (2025)
- [UniAct: Unified Motion Generation and Action Streaming for Humanoid Robots](https://mlsanity.com/paper/2512.24321/) (2025)
- [No More Blind Spots: Learning Vision-Based Omnidirectional Bipedal Locomotion for Challenging Terrain](https://mlsanity.com/paper/2508.11929/) (2025)
- [Global-Local Attention Decomposition for Terrain Encoding in Humanoid Perceptive Locomotion](https://mlsanity.com/paper/2606.00637/) (2026)
- [Robust RL Control for Bipedal Locomotion with Closed Kinematic Chains](https://mlsanity.com/paper/2507.10164/) (2025)
- [Sim-to-Real of Humanoid Locomotion Policies via Joint Torque Space Perturbation Injection](https://mlsanity.com/paper/2603.21853/) (2026)
- [CoRe-MoE: Contrastive Reweighted Mixture of Experts for Multi-Terrain Humanoid Locomotion with Gait Adaptation](https://mlsanity.com/paper/2606.04718/) (2026)
- [Semantic Audio-driven Understanding for Dynamic Humanoid Whole Body Control](https://mlsanity.com/paper/2607.14182/) (2026)
- [Loco-Loco-RL: Low-Cost Terrain Mapping for Humanoid Locomotion with Reinforcement Learning](https://mlsanity.com/paper/2609.19041/) (2026)
- [Learning Slope-Adaptive Whole-Body Locomotion for Humanoid Robots in Roofing Construction](https://mlsanity.com/paper/2609.20558/) (2026)
- [Smoothness as a Constraint for Stable Humanoid Locomotion](https://mlsanity.com/paper/2609.24552/) (2026)

## Abstract (arXiv)

Humanoid robots can, in principle, use their legs to go almost anywhere. Developing controllers capable of traversing diverse terrains, however, remains a considerable challenge. Classical controllers are hard to generalize broadly while the learning-based methods have primarily focused on gentle terrains. Here, we present a learning-based approach for blind humanoid locomotion capable of traversing challenging natural and man-made terrain. Our method uses a transformer model to predict the next action based on the history of proprioceptive observations and actions. The model is first pre-trained on a dataset of flat-ground trajectories with sequence modeling, and then fine-tuned on uneven terrain using reinforcement learning. We evaluate our model on a real humanoid robot across a variety of terrains, including rough, deformable, and sloped surfaces. The model demonstrates robust performance, in-context adaptation, and emergent terrain representations. In real-world case studies, our humanoid robot successfully traversed over 4 miles of hiking trails in Berkeley and climbed some of the steepest streets in San Francisco.