# Data Efficient Reinforcement Learning for Legged Robots

arXiv: https://arxiv.org/abs/1907.03613 · HTML: https://mlsanity.com/paper/1907.03613/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jul 2019.

## Built on by (most cited first, 30 of 63)

- [MOReL : Model-Based Offline Reinforcement Learning](https://mlsanity.com/paper/2005.05951/) (2020)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [DayDreamer: World Models for Physical Robot Learning](https://mlsanity.com/paper/2206.14176/) (2022)
- [How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned](https://mlsanity.com/paper/2102.02915/) (2021)
- [SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning](https://mlsanity.com/paper/2401.16013/) (2024)
- [A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learning](https://mlsanity.com/paper/2208.07860/) (2022)
- [Fast and Efficient Locomotion via Learned Gait Transitions](https://mlsanity.com/paper/2104.04644/) (2021)
- [Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics](https://mlsanity.com/paper/2501.10100/) (2025)
- [Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classification](https://mlsanity.com/paper/2502.02133/) (2025)
- [GLiDE: Generalizable Quadrupedal Locomotion in Diverse Environments with a Centroidal Model](https://mlsanity.com/paper/2104.09771/) (2021)
- [Grow Your Limits: Continuous Improvement with Real-World RL for Robotic Locomotion](https://mlsanity.com/paper/2310.17634/) (2023)
- [Vision-Guided Quadrupedal Locomotion in the Wild with Multi-Modal Delay Randomization](https://mlsanity.com/paper/2109.14549/) (2021)
- [Learning Modular Robot Control Policies](https://mlsanity.com/paper/2105.10049/) (2021)
- [Uncertainty-Aware Robotic World Model Makes Offline Model-Based Reinforcement Learning Work on Real Robots](https://mlsanity.com/paper/2504.16680/) (2025)
- [Learning to Walk from Three Minutes of Real-World Data with Semi-structured Dynamics Models](https://mlsanity.com/paper/2410.09163/) (2024)
- [A Relational Intervention Approach for Unsupervised Dynamics Generalization in Model-Based Reinforcement Learning](https://mlsanity.com/paper/2206.04551/) (2022)
- [Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Models](https://mlsanity.com/paper/2407.02666/) (2024)
- [Regret Lower Bounds for Learning Linear Quadratic Gaussian Systems](https://mlsanity.com/paper/2201.01680/) (2022)
- [Inferring Smooth Control: Monte Carlo Posterior Policy Iteration with Gaussian Processes](https://mlsanity.com/paper/2210.03512/) (2022)
- [Dexterous Legged Locomotion in Confined 3D Spaces with Reinforcement Learning](https://mlsanity.com/paper/2403.03848/) (2024)
- [Video2Reward: Generating Reward Function from Videos for Legged Robot Behavior Learning](https://mlsanity.com/paper/2412.05515/) (2024)
- [Bridging Active Exploration and Uncertainty-Aware Deployment Using Probabilistic Ensemble Neural Network Dynamics](https://mlsanity.com/paper/2305.12240/) (2023)
- [Deep Whole-body Parkour](https://mlsanity.com/paper/2601.07701/) (2026)
- [Cloud-Edge Training Architecture for Sim-to-Real Deep Reinforcement Learning](https://mlsanity.com/paper/2203.02230/) (2022)
- [World Robot Challenge 2020 -- Partner Robot: A Data-Driven Approach for Room Tidying with Mobile Manipulator](https://mlsanity.com/paper/2207.10106/) (2022)
- [Minimizing Human Assistance: Augmenting a Single Demonstration for Deep Reinforcement Learning](https://mlsanity.com/paper/2209.11275/) (2022)
- [A Simple Decentralized Cross-Entropy Method](https://mlsanity.com/paper/2212.08235/) (2022)
- [Safe Navigation in Unstructured Environments by Minimizing Uncertainty in Control and Perception](https://mlsanity.com/paper/2306.14601/) (2023)
- [TAR: Teacher-Aligned Representations via Contrastive Learning for Quadrupedal Locomotion](https://mlsanity.com/paper/2503.20839/) (2025)
- [Simulation-Aided Policy Tuning for Black-Box Robot Learning](https://mlsanity.com/paper/2411.14246/) (2024)

## Abstract (arXiv)

We present a model-based framework for robot locomotion that achieves walking based on only 4.5 minutes (45,000 control steps) of data collected on a quadruped robot. To accurately model the robot's dynamics over a long horizon, we introduce a loss function that tracks the model's prediction over multiple timesteps. We adapt model predictive control to account for planning latency, which allows the learned model to be used for real time control. Additionally, to ensure safe exploration during model learning, we embed prior knowledge of leg trajectories into the action space. The resulting system achieves fast and robust locomotion. Unlike model-free methods, which optimize for a particular task, our planner can use the same learned dynamics for various tasks, simply by changing the reward function. To the best of our knowledge, our approach is more than an order of magnitude more sample efficient than current model-free methods.