mlsanityData to 7 Oct 2026

Real-World Humanoid Locomotion with Reinforcement Learning

arXiv 2303.03381 ↗First posted 6 Mar 2023cs.RO cs.LGcited by 132

Not on the map. Why?

Abstract (arXiv)

Humanoid robots that can autonomously operate in diverse environments have the potential to help address labour shortages in factories, assist elderly at homes, and colonize new planets. While classical controllers for humanoid robots have shown impressive results in a number of settings, they are challenging to generalize and adapt to new environments. Here, we present a fully learning-based approach for real-world humanoid locomotion. Our controller is a causal transformer that takes the history of proprioceptive observations and actions as input and predicts the next action. We hypothesize that the observation-action history contains useful information about the world that a powerful transformer model can use to adapt its behavior in-context, without updating its weights. We train our model with large-scale model-free reinforcement learning on an ensemble of randomized environments in simulation and deploy it to the real world zero-shot. Our controller can walk over various outdoor terrains, is robust to external disturbances, and can adapt in context.

Key facts. Cited by 132 papers in the map (data through 7 Oct 2026). Builds on 35 of its 62 references that are in the map. First posted on arXiv 6 Mar 2023.

Builds on

35 of 62 references.

References · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2017Proximal Policy Optimization Algorithmscited by 14,441
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2020Scaling Laws for Neural Language Modelscited by 6,411
  8. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  9. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,447
  10. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  11. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  12. 2019Solving Rubik's Cube with a Robot Handcited by 699
23 more
  1. 2018Learning Dexterous In-Hand Manipulationcited by 643
  2. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  3. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 494
  4. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 430
  5. 2019Learning agile and dynamic motor skills for legged robotscited by 402
  6. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 395
  7. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 277
  8. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 227
  9. 2022DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Realitycited by 136
  10. 2021Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learningcited by 73
  11. 2020Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Compositioncited by 71
  12. 2018Feedback Control For Cassie With Deep Reinforcement Learningcited by 36
  13. 2021The MIT Humanoid Robot: Design, Motion Planning, and Control For Acrobatic Behaviorscited by 28
  14. 2017Reinforcement Learning for Pivoting Taskcited by 17
  15. 2018Feedback Control of a Cassie Bipedal Robot: Walking, Standing, and Riding a Segwaycited by 15
  16. 2021DeepWalk: Omnidirectional Bipedal Gait by Deep Reinforcement Learningcited by 15
  17. 2021Robust Feedback Motion Policy Design Using Reinforcement Learning on a 3D Digit Bipedal Robotcited by 12
  18. 2021Linear Policies are Sufficient to Realize Robust Bipedal Walking on Challenging Terrainscited by 7
  19. 2021Integrated Task and Motion Planning for Safe Legged Navigation in Partially Observable Environmentscited by 5
  20. 2022Optimal Constrained Task Planning as Mixed Integer Programmingcited by 3
  21. 2022Invariant Filtering for Legged Humanoid Locomotion on Dynamic Rigid Surfacescited by 2
  22. 2022A Sequential MPC Approach to Reactive Planning for Bipedal Robotscited by 2
  23. 2022Time-Varying ALIP Model and Robust Foot-Placement Control for Underactuated Bipedal Robot Walking on a Swaying Rigid Surfacecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 32; 2025: 56; 2026: 38.

132 papers.

Sort:

By area:

  1. 2024OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learningcited by 167
  2. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 153
  3. 2025BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusioncited by 128
  4. 2024Expressive Whole-Body Control for Humanoid Robotscited by 117
  5. 2025ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skillscited by 104
  6. 2025TWIST: Teleoperated Whole-Body Imitation Systemcited by 86
  7. 2024ExBody2: Advanced Expressive Humanoid Whole-Body Controlcited by 75
  8. 2024Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Controlcited by 63
  9. 2024Humanoid Locomotion as Next Token Predictioncited by 63
  10. 2025Visual Imitation Enables Contextual Humanoid Controlcited by 57
  11. 2024PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful Navigatorscited by 55
  12. 2024TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correctioncited by 50
38 more
  1. 2024Generalizable Humanoid Manipulation with 3D Diffusion Policiescited by 49
  2. 2025Residual Off-Policy RL for Finetuning Behavior Cloning Policiescited by 42
  3. 2025Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoidscited by 40
  4. 2024One Policy to Run Them All: an End-to-end Learning Approach to Multi-Embodiment Locomotioncited by 34
  5. 2025Learning Humanoid Standing-up Control across Diverse Posturescited by 34
  6. 2025ResMimic: From General Motion Tracking to Humanoid Whole-body Loco-Manipulation via Residual Learningcited by 34
  7. 2023Not Only Rewards But Also Constraints: Applications on Legged Robot Locomotioncited by 33
  8. 2025What Is The Best 3D Scene Representation for Robotics? From Geometric to Foundation Modelscited by 29
  9. 2024Learning from Massive Human Videos for Universal Humanoid Pose Controlcited by 27
  10. 2024Learning Smooth Humanoid Locomotion through Lipschitz-Constrained Policiescited by 26
  11. 2025The Reality Gap in Robotics: Challenges, Solutions, and Best Practicescited by 22
  12. 2025Learning Getting-Up Policies for Real-World Humanoid Robotscited by 21
  13. 2025Being-0: A Humanoid Robotic Agent with Vision-Language Models and Modular Skillscited by 21
  14. 2025HuB: Learning Extreme Humanoid Balancecited by 21
  15. 2024Learning Humanoid Locomotion over Challenging Terraincited by 20
  16. 2024A Survey of Optimization-based Task and Motion Planning: From Classical To Learning Approachescited by 19
  17. 2024The One RING: a Robotic Indoor Navigation Generalistcited by 15
  18. 2024Learning Robot Soccer from Egocentric Vision with Deep Reinforcement Learningcited by 14
  19. 2024Stabilizing Reinforcement Learning in Differentiable Multiphysics Simulationcited by 14
  20. 2025LocoFormer: Generalist Locomotion via Long-context Adaptationcited by 14
  21. 2025DreamControl: Human-Inspired Whole-Body Humanoid Control for Scene Interaction via Guided Diffusioncited by 13
  22. 2025Towards Embodiment Scaling Laws in Robot Locomotioncited by 12
  23. 2025Bridging the Sim-to-Real Gap for Athletic Loco-Manipulationcited by 11
  24. 2025KungfuBot2: Learning Versatile Motion Skills for Humanoid Whole-Body Controlcited by 11
  25. 2026Hiking in the Wild: A Scalable Perceptive Parkour Framework for Humanoidscited by 11
  26. 2025A Unified and General Humanoid Whole-Body Controller for Versatile Locomotioncited by 10
  27. 2025Learning Humanoid Locomotion with World Model Reconstructioncited by 10
  28. 2025ArticuBot: Learning Universal Articulated Object Manipulation Policy via Large Scale Simulationcited by 10
  29. 2025Physical Plausibility-aware Trajectory Prediction via Locomotion Embodimentcited by 10
  30. 2026HEX: Humanoid-Aligned Experts for Cross-Embodiment Whole-Body Manipulationcited by 10
  31. 2023Towards Generalist Robots: A Promising Paradigm via Generative Simulationcited by 9
  32. 2023Learning Bipedal Walking for Humanoids with Current Feedbackcited by 8
  33. 2024Feedback Favors the Generalization of Neural ODEscited by 8
  34. 2024HYPERmotion: Learning Hybrid Behavior Planning for Autonomous Loco-manipulationcited by 7
  35. 2025A roadmap for AI in roboticscited by 7
  36. 2024Student-Informed Teacher Trainingcited by 6
  37. 2025Relative Entropy Pathwise Policy Optimizationcited by 6
  38. 2025Towards Adaptable Humanoid Control via Adaptive Motion Trackingcited by 6

50 most cited of 132 All 132 (JSON)