mlsanityData to 14 Oct 2026

Learning to Walk in the Real World with Minimal Human Effort

arXiv 2002.08550 ↗First posted 20 Feb 2020cs.RO cs.AI cs.LGcited by 111

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

Reliable and stable locomotion has been one of the most fundamental challenges for legged robots. Deep reinforcement learning (deep RL) has emerged as a promising method for developing such control policies autonomously. In this paper, we develop a system for learning legged locomotion policies with deep RL in the real world with minimal human effort. The key difficulties for on-robot learning systems are automatic data collection and safety. We overcome these two challenges by developing a multi-task learning procedure and a safety-constrained RL framework. We tested our system on the task of learning to walk on three different terrains: flat ground, a soft mattress, and a doormat with crevices. Our system can automatically and efficiently learn locomotion skills on a Minitaur robot with little human intervention. The supplemental video can be found at: urlhttps://youtu.be/cwyiq6dCgOc.

Key facts. Cited by 111 papers in the map (data through 14 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 4 of its 67 references that are in the map. First posted on arXiv 20 Feb 2020.

Builds on

4 of 67 references.

Closest ancestors · 2

Same direction, newest first

  1. 2019When to Trust Your Model: Model-Based Policy Optimization19 Jun
  2. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models30 May

Other references · 2

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2019Solving Rubik's Cube with a Robot Handcited by 734

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 13; 2022: 15; 2023: 23; 2024: 23; 2025: 17; 2026: 19.

111 papers.

Sort:

By area:

  1. 2022DayDreamer: World Models for Physical Robot Learningcited by 400
  2. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 290
  3. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 166
  4. 2023Robot Parkour Learningcited by 149
  5. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 133
  6. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 126
  7. 2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 95
  8. 2021Constraints Penalized Q-learning for Safe Offline Reinforcement Learningcited by 91
  9. 2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learningcited by 83
  10. 2023Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learningcited by 54
  11. 2022Efficient Learning of Safe Driving Policy via Human-AI Copilot Optimizationcited by 49
  12. 2022Safe Reinforcement Learning for Legged Locomotioncited by 35
38 more
  1. 2021Autonomous Reinforcement Learning via Subgoal Curriculacited by 33
  2. 2021Autonomous Reinforcement Learning: Formalism and Benchmarkingcited by 33
  3. 2024HAIM-DRL: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Drivingcited by 32
  4. 2021Safe Driving via Expert Guided Policy Optimizationcited by 31
  5. 2023DiffuseBot: Breeding Soft Robots With Physics-Augmented Generative Diffusion Modelscited by 31
  6. 2022Learning Torque Control for Quadrupedal Locomotioncited by 29
  7. 2022Don't Start From Scratch: Leveraging Prior Data to Automate Robotic Reinforcement Learningcited by 29
  8. 2023Grow Your Limits: Continuous Improvement with Real-World RL for Robotic Locomotioncited by 24
  9. 2021Learning Modular Robot Control Policiescited by 22
  10. 2022A State-Distribution Matching Approach to Non-Episodic Reinforcement Learningcited by 22
  11. 2022Evaluating Model-free Reinforcement Learning toward Safety-critical Taskscited by 22
  12. 2024Continuously Improving Mobile Manipulation with Autonomous Real-World RLcited by 22
  13. 2025Learning from Active Human Involvement through Proxy Value Propagationcited by 19
  14. 2023FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicingcited by 17
  15. 2020Lipschitzness Is All You Need To Tame Off-policy Generative Adversarial Imitation Learningcited by 15
  16. 2024Constrained Reinforcement Learning with Smoothed Log Barrier Functioncited by 15
  17. 2022Automating Reinforcement Learning with Example-based Resetscited by 12
  18. 2022Safe Reinforcement Learning From Pixels Using a Stochastic Latent Representationcited by 11
  19. 2023Efficient Off-Policy Safe Reinforcement Learning Using Trust Region Conditional Value at Riskcited by 11
  20. 2024DDM-Lag : A Diffusion-based Decision-making Model for Autonomous Vehicles with Lagrangian Safety Enhancementcited by 11
  21. 2022Cascaded Compositional Residual Learning for Complex Interactive Behaviorscited by 10
  22. 2023Safe Reinforcement Learning with Dual Robustnesscited by 10
  23. 2022Deep reinforcement learning under signal temporal logic constraints using Lagrangian relaxationcited by 8
  24. 2023Residual Physics Learning and System Identification for Sim-to-real Transfer of Policies on Buoyancy Assisted Legged Robotscited by 8
  25. 2024Trustworthy Human-AI Collaboration: Reinforcement Learning with Human Feedback and Physics Knowledge for Safe Autonomous Drivingcited by 8
  26. 2021Improving Safety in Deep Reinforcement Learning using Unsupervised Action Planningcited by 7
  27. 2022Learning to Walk Autonomously via Reset-Free Quality-Diversitycited by 7
  28. 2022SafeRL-Kit: Evaluating Efficient Reinforcement Learning Methods for Safe Autonomous Drivingcited by 7
  29. 2023Reinforcement Learning by Guided Safe Explorationcited by 7
  30. 2025Robot Trains Robot: Automatic Real-World Policy Adaptation and Learning for Humanoidscited by 7
  31. 2021Towards Disturbance-Free Visual Mobile Manipulationcited by 6
  32. 2023Reduced Policy Optimization for Continuous Control with Hard Constraintscited by 6
  33. 2024Knowledge Transfer from Simple to Complex: A Safe and Efficient Reinforcement Learning Framework for Autonomous Driving Decision-Makingcited by 6
  34. 2024Safe Multi-Agent Reinforcement Learning with Convergence to Generalized Nash Equilibriumcited by 6
  35. 2021Residual Model Learning for Microrobot Controlcited by 5
  36. 2021Deep Reinforcement Learning for Conservation Decisionscited by 5
  37. 2023Safe Reinforcement Learning with Dead-Ends Avoidance and Recoverycited by 5
  38. 2023Prioritized Soft Q-Decomposition for Lexicographic Reinforcement Learningcited by 5

50 most cited of 111 All 111 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Robust Humanoid Walking on Compliant and Uneven Terrain with Deep Reinforcement Learning
  2. 2024Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control
  3. 2025Whole-Body Constrained Learning for Legged Locomotion via Hierarchical Optimization
  4. 2024Deep Reinforcement Learning for Bipedal Locomotion: A Brief Survey
  5. 2024Dexterous Legged Locomotion in Confined 3D Spaces with Reinforcement Learning
  6. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning
  7. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes
  8. 2025Taxonomy and Trends in Reinforcement Learning for Robotics and Control Systems: A Structured Review