mlsanityData to 14 Oct 2026

Learning to Walk via Deep Reinforcement Learning

arXiv 1812.11103 ↗First posted 26 Dec 2018cs.LG cs.AI cs.ROcited by 158

Not on the map. Why?

Cited by 31 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Deep reinforcement learning (deep RL) holds the promise of automating the acquisition of complex controllers that can map sensory inputs directly to low-level actions. In the domain of robotic locomotion, deep RL could enable learning locomotion skills with minimal engineering and without an explicit model of the robot dynamics. Unfortunately, applying deep RL to real-world robotic tasks is exceptionally difficult, primarily due to poor sample complexity and sensitivity to hyperparameters. While hyperparameters can be easily tuned in simulated domains, tuning may be prohibitively expensive on physical systems, such as legged robots, that can be damaged through extensive trial-and-error learning. In this paper, we propose a sample-efficient deep RL algorithm based on maximum entropy RL that requires minimal per-task tuning and only a modest number of trials to learn neural network policies. We apply this method to learning walking gaits on a real-world Minitaur robot. Our method can acquire a stable gait from scratch directly in the real world in about two hours, without relying on any model or simulation, and the resulting policy is robust to moderate variations in the environment. We further show that our algorithm achieves state-of-the-art performance on simulated benchmarks with a single set of hyperparameters. Videos of training and the learned policy can be found on the project website.

Key facts. Cited by 158 papers in the map (data through 14 Oct 2026). First posted on arXiv 26 Dec 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 42; 2022: 31; 2023: 32; 2024: 25; 2025: 16; 2026: 11.

158 papers.

Sort:

By area:

  1. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
  2. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 213
  3. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 176
  4. 2021BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environmentscited by 165
  5. 2023Robot Parkour Learningcited by 149
  6. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 133
  7. 2024Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Controlcited by 99
  8. 2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learningcited by 83
  9. 2022Latent Plans for Task-Agnostic Offline Reinforcement Learningcited by 81
  10. 2021Minimizing Energy Consumption Leads to the Emergence of Gaits in Legged Robotscited by 76
  11. 2021Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordinationcited by 70
  12. 2024PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful Navigatorscited by 58
38 more
  1. 2022Adapting Rapid Motor Adaptation for Bipedal Robotscited by 54
  2. 2022GenLoco: Generalized Locomotion Controllers for Quadrupedal Robotscited by 48
  3. 2019Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreementcited by 46
  4. 2026ENPIRE: Agentic Robot Policy Self-Improvement in the Real Worldcited by 38
  5. 2023Robust and Versatile Bipedal Jumping Control through Reinforcement Learningcited by 36
  6. 2021Fully Autonomous Real-World Reinforcement Learning with Applications to Mobile Manipulationcited by 35
  7. 2022Safe Reinforcement Learning for Legged Locomotioncited by 35
  8. 2021Lifelong Robotic Reinforcement Learning by Retaining Experiencescited by 34
  9. 2023Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learningcited by 34
  10. 2021Autonomous Reinforcement Learning via Subgoal Curriculacited by 33
  11. 2023Learning and Adapting Agile Locomotion Skills by Transferring Experiencecited by 33
  12. 2021Regularized Policies are Reward Robustcited by 32
  13. 2022Imitate and Repurpose: Learning Reusable Robot Movement Skills From Human and Animal Behaviorscited by 30
  14. 2022Learning Torque Control for Quadrupedal Locomotioncited by 29
  15. 2021A Max-Min Entropy Framework for Reinforcement Learningcited by 28
  16. 2021Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Surveycited by 28
  17. 2022Active Exploration for Inverse Reinforcement Learningcited by 27
  18. 2021Latent Imagination Facilitates Zero-Shot Transfer in Autonomous Racingcited by 26
  19. 2022Asynchronous Reinforcement Learning for Real-Time Control of Physical Robotscited by 22
  20. 2022Humans are not Boltzmann Distributions: Challenges and Opportunities for Modelling Human Feedback and Interaction in Reinforcement Learningcited by 22
  21. 2021Bayesian Controller Fusion: Leveraging Control Priors in Deep Reinforcement Learning for Roboticscited by 20
  22. 2022DHRL: A Graph-Based Approach for Long-Horizon and Sparse Hierarchical Reinforcement Learningcited by 19
  23. 2023TWIST: Teacher-Student World Model Distillation for Efficient Sim-to-Real Transfercited by 19
  24. 2021Imitation Learning from MPC for Quadrupedal Multi-Gait Controlcited by 18
  25. 2024Continuous Control with Coarse-to-fine Reinforcement Learningcited by 17
  26. 2025Learning Sim-to-Real Humanoid Locomotion in 15 Minutescited by 16
  27. 2024Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Modelscited by 15
  28. 2022Real-Time Reinforcement Learning for Vision-Based Robotics Utilizing Local and Remote Computerscited by 13
  29. 2023Robotic Table Tennis: A Case Study into a High Speed Learning Systemcited by 13
  30. 2021Cooperative Multi-Agent Transfer Learning with Level-Adaptive Credit Assignmentcited by 12
  31. 2021Exploration and preference satisfaction trade-off in reward-free learningcited by 12
  32. 2021The Role of Pretrained Representations for the OOD Generalization of Reinforcement Learning Agentscited by 12
  33. 2023Diffused Task-Agnostic Milestone Plannercited by 12
  34. 2021Meta-Reinforcement Learning in Broad and Non-Parametric Environmentscited by 10
  35. 2021Addressing Action Oscillations through Learning Policy Inertiacited by 9
  36. 2021Optimizing Quantum Variational Circuits with Deep Reinforcement Learningcited by 9
  37. 2023Maximum diffusion reinforcement learningcited by 9
  38. 2021Full Gradient DQN Reinforcement Learning: A Provably Convergent Schemecited by 8

50 most cited of 158 All 158 (JSON)