Data to 14 Oct 2026

A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learning

arXiv 2208.07860 ↗First posted 16 Aug 2022cs.RO cs.AIcited by 83

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

Deep reinforcement learning is a promising approach to learning policies in uncontrolled environments that do not require domain knowledge. Unfortunately, due to sample inefficiency, deep RL applications have primarily focused on simulated environments. In this work, we demonstrate that the recent advancements in machine learning algorithms and libraries combined with a carefully tuned robot controller lead to learning quadruped locomotion in only 20 minutes in the real world. We evaluate our approach on several indoor and outdoor terrains which are known to be challenging for classical model-based controllers. We observe the robot to be able to learn walking gait consistently on all of these terrains. Finally, we evaluate our design decisions in a simulated environment.

Key facts. Cited by 83 papers in the map (data through 14 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 47 of its 69 references that are in the map. First posted on arXiv 16 Aug 2022.

Builds on

47 of 69 references.

Closest ancestors · 4

Same direction, newest first

Other references · 43

2016Layer Normalizationcited by 5,669
2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
2019Solving Rubik's Cube with a Robot Handcited by 734
2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
2019Learning agile and dynamic motor skills for legged robotscited by 504
2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389

31 more

2016Learning to reinforcement learncited by 377
2020dm_control: Software and Tasks for Continuous Controlcited by 375
2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 353
2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 319
2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 316
2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 288
2019RoboNet: Large-Scale Multi-Robot Learningcited by 270
2020Learning Agile Robotic Locomotion Skills by Imitating Animalscited by 251
2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 250
2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 233
2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 229
2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 217
2018Learning to Walk via Deep Reinforcement Learningcited by 158
2016EPOpt: Learning Robust Neural Network Policies Using Model Ensemblescited by 138
2021Dropout Q-Functions for Doubly Efficient Reinforcement Learningcited by 114
2015Massively Parallel Methods for Deep Reinforcement Learningcited by 82
2021Minimizing Energy Consumption Leads to the Emergence of Gaits in Legged Robotscited by 76
2021Legged Robots that Keep on Learning: Fine-Tuning Locomotion Policies in the Real Worldcited by 71
2019Data Efficient Reinforcement Learning for Legged Robotscited by 63
2019Sim-to-Real Transfer for Biped Locomotioncited by 56
2021Fast and Efficient Locomotion via Learned Gait Transitionscited by 54
2019Policies Modulating Trajectory Generatorscited by 49
2021Auto-Tuned Sim-to-Real Transfercited by 49
2016Reinforcement Learning through Asynchronous Advantage Actor-Critic on a GPUcited by 48
2019Learning Fast Adaptation with Meta Strategy Optimizationcited by 48
2020Rapidly Adaptable Legged Robots via Evolutionary Meta-Learningcited by 43
2022Safe Reinforcement Learning for Legged Locomotioncited by 35
2019Robust Recovery Controller for a Quadrupedal Robot using Deep Reinforcement Learningcited by 32
2019Asynchronous Methods for Model-Based Reinforcement Learningcited by 7
2018Simulator Predictive Control: Using Learned Task Representations and MPC for Zero-Shot Generalization and Sequencingcited by 4
2017From the Lab to the Desert: Fast Prototyping and Learning of Robot Locomotioncited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 5; 2023: 23; 2024: 23; 2025: 16; 2026: 16.

83 papers.

Sort:

By area:

2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 571
2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 166
2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 133
2023Revisiting the Minimalist Approach to Offline Reinforcement Learningcited by 126
2022CORL: Research-oriented Deep Offline Reinforcement Learning Librarycited by 119
2024Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviationcited by 114
2022Holo-Dex: Teaching Dexterity with Immersive Mixed Realitycited by 57
2023Anti-Exploration by Random Network Distillationcited by 50
2024Overestimation, Overfitting, and Plasticity in Actor-Critic: the Bitter Lesson of Reinforcement Learningcited by 50
2022Retrospectives on the Embodied AI Workshopcited by 46
2024One Policy to Run Them All: an End-to-end Learning Approach to Multi-Embodiment Locomotioncited by 43
2023Robust and Versatile Bipedal Jumping Control through Reinforcement Learningcited by 36

38 more

2023Efficient Deep Reinforcement Learning Requires Regulating Overfittingcited by 36
2023Learning and Adapting Agile Locomotion Skills by Transferring Experiencecited by 33
2024Rank2Reward: Learning Shaped Reward Functions from Passive Videocited by 32
2022Choreographer: Learning and Adapting Skills in Imaginationcited by 31
2023RoboPianist: Dexterous Piano Playing with Deep Reinforcement Learningcited by 30
2024MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximizationcited by 30
2023Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stagescited by 28
2023Uni-O4: Unifying Online and Offline Deep Reinforcement Learning with Multi-Step On-Policy Optimizationcited by 26
2023On the Role of the Action Space in Robot Manipulation Learning and Sim-to-Real Transfercited by 26
2024CaT: Constraints as Terminations for Legged Locomotion Reinforcement Learningcited by 23
2025LocoFormer: Generalist Locomotion via Long-context Adaptationcited by 20
2024Continuous Control with Coarse-to-fine Reinforcement Learningcited by 17
2025Rapidly Adapting Policies to the Real World via Simulation-Guided Fine-Tuningcited by 17
2025Uncertainty-Aware Robotic World Model Makes Offline Model-Based Reinforcement Learning Work on Real Robotscited by 17
2024Learning to Walk from Three Minutes of Real-World Data with Semi-structured Dynamics Modelscited by 16
2025Towards Embodiment Scaling Laws in Robot Locomotioncited by 16
2024Constrained Reinforcement Learning with Smoothed Log Barrier Functioncited by 15
2024SoloParkour: Constrained Reinforcement Learning for Visual Locomotion from Privileged Experiencecited by 14
2025Touch begins where vision ends: Generalizable policies for contact-rich manipulationcited by 14
2023Robotic Table Tennis: A Case Study into a High Speed Learning Systemcited by 13
2023Autonomous Robotic Reinforcement Learning with Asynchronous Human Feedbackcited by 13
2024D5RL: Diverse Datasets for Data-Driven Deep Reinforcement Learningcited by 12
2025Simplicial Embeddings Improve Sample Efficiency in Actor-Critic Agentscited by 12
2026Large Reward Models: Generalizable Online Robot Reward Generation with Vision-Language Modelscited by 11
2025Subtask-Aware Visual Reward Learning from Segmented Demonstrationscited by 10
2023Mind the Model, Not the Agent: The Primacy Bias in Model-based RLcited by 9
2025Sketch-to-Skill: Bootstrapping Robot Learning with Human Drawn Trajectory Sketchescited by 9
2023Learning Robust, Agile, Natural Legged Locomotion Skills in the Wildcited by 8
2023DexCatch: Learning to Catch Arbitrary Objects with Dexterous Handscited by 8
2023Adapt On-the-Go: Behavior Modulation for Single-Life Robot Deploymentcited by 8
2024Robot Policy Learning with Temporal Optimal Transport Rewardcited by 8
2025Gait in Eight: Efficient On-Robot Learning for Omnidirectional Quadruped Locomotioncited by 8
2023Symmetric Replay Training: Enhancing Sample Efficiency in Deep Reinforcement Learning for Combinatorial Optimizationcited by 7
2025SLAC: Safe and Efficient Real-Robot Reinforcement Learning via Unsupervised Simulation Pre-Trainingcited by 7
2025Robot Trains Robot: Automatic Real-World Policy Adaptation and Learning for Humanoidscited by 7
2023On the Theory of Risk-Aware Agents: Bridging Actor-Critic and Economicscited by 6
2026Simulation Distillation: Pretraining World Models in Simulation for Rapid Real-World Adaptationcited by 6
2023Katakomba: Tools and Benchmarks for Data-Driven NetHackcited by 5

50 most cited of 83 All 83 (JSON)

Similar, not cited

Close in wording, no citation either way