mlsanityData to 14 Oct 2026

Legged Robots that Keep on Learning: Fine-Tuning Locomotion Policies in the Real World

arXiv 2110.05457 ↗First posted 11 Oct 2021cs.ROcited by 71

Not on the map. Why?

Cited by 25 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Legged robots are physically capable of traversing a wide range of challenging environments, but designing controllers that are sufficiently robust to handle this diversity has been a long-standing challenge in robotics. Reinforcement learning presents an appealing approach for automating the controller design process and has been able to produce remarkably robust controllers when trained in a suitable range of environments. However, it is difficult to predict all likely conditions the robot will encounter during deployment and enumerate them at training-time. What if instead of training controllers that are robust enough to handle any eventuality, we enable the robot to continually learn in any setting it finds itself in? This kind of real-world reinforcement learning poses a number of challenges, including efficiency, safety, and autonomy. To address these challenges, we propose a practical robot reinforcement learning system for fine-tuning locomotion policies in the real world. We demonstrate that a modest amount of real-world training can substantially improve performance during deployment, and this enables a real A1 quadrupedal robot to autonomously fine-tune multiple locomotion skills in a range of environments, including an outdoor lawn and a variety of indoor terrains.

Key facts. Cited by 71 papers in the map (data through 14 Oct 2026). First posted on arXiv 11 Oct 2021.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 11; 2023: 16; 2024: 17; 2025: 8; 2026: 18.

71 papers.

Sort:

By area:

  1. 2022DayDreamer: World Models for Physical Robot Learningcited by 400
  2. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 176
  3. 2023Extreme Parkour with Legged Robotscited by 164
  4. 2022Legged Locomotion in Challenging Terrains using Egocentric Visioncited by 151
  5. 2023Robot Parkour Learningcited by 149
  6. 2022Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotioncited by 121
  7. 2024Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Controlcited by 99
  8. 2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learningcited by 83
  9. 2024DoraemonGPT: Toward Understanding Dynamic Scenes with Large Language Models (Exemplified as A Video Agent)cited by 58
  10. 2021Coupling Vision and Proprioception for Navigation of Legged Robotscited by 51
  11. 2022GenLoco: Generalized Locomotion Controllers for Quadrupedal Robotscited by 48
  12. 2023Prompt a Robot to Walk with Large Language Modelscited by 45
38 more
  1. 2023Barkour: Benchmarking Animal-level Agility with Quadruped Robotscited by 41
  2. 2023DribbleBot: Dynamic Legged Manipulation in the Wildcited by 40
  3. 2022Hierarchical Reinforcement Learning for Precise Soccer Shooting Skills using a Quadrupedal Robotcited by 38
  4. 2023Robust and Versatile Bipedal Jumping Control through Reinforcement Learningcited by 36
  5. 2023Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learningcited by 34
  6. 2024Autonomous Improvement of Instruction Following Skills via Foundation Modelscited by 32
  7. 2024Adaptive Mobile Manipulation for Articulated Objects In the Open Worldcited by 31
  8. 2022Learning Torque Control for Quadrupedal Locomotioncited by 29
  9. 2024ASID: Active Exploration for System Identification in Robotic Manipulationcited by 28
  10. 2023Uni-O4: Unifying Online and Offline Deep Reinforcement Learning with Multi-Step On-Policy Optimizationcited by 26
  11. 2022A State-Distribution Matching Approach to Non-Episodic Reinforcement Learningcited by 22
  12. 2024Continuously Improving Mobile Manipulation with Autonomous Real-World RLcited by 22
  13. 2023Dynamics Generalisation in Reinforcement Learning via Adaptive Context-Aware Policiescited by 20
  14. 2024Learning Visual Quadrupedal Loco-Manipulation from Demonstrationscited by 20
  15. 2025Rapidly Adapting Policies to the Real World via Simulation-Guided Fine-Tuningcited by 17
  16. 2024Learning to Walk from Three Minutes of Real-World Data with Semi-structured Dynamics Modelscited by 16
  17. 2024Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Modelscited by 15
  18. 2023Learning Risk-Aware Quadrupedal Locomotion using Distributional Reinforcement Learningcited by 14
  19. 2024SoloParkour: Constrained Reinforcement Learning for Visual Locomotion from Privileged Experiencecited by 14
  20. 2023Robotic Table Tennis: A Case Study into a High Speed Learning Systemcited by 13
  21. 2024Stage-Wise Reward Shaping for Acrobatic Robots: A Constrained Multi-Objective Reinforcement Learning Approachcited by 13
  22. 2023ElasticTrainer: Speeding Up On-Device Training with Runtime Elastic Tensor Selectioncited by 12
  23. 2022Cascaded Compositional Residual Learning for Complex Interactive Behaviorscited by 10
  24. 2025From Screens to Scenes: A Survey of Embodied AI in Healthcarecited by 10
  25. 2024Helpful DoggyBot: Open-World Object Fetching using Legged Robots and Vision-Language Modelscited by 8
  26. 2023Identifying Important Sensory Feedback for Learning Locomotion Skillscited by 6
  27. 2026Simulation Distillation: Pretraining World Models in Simulation for Rapid Real-World Adaptationcited by 6
  28. 2024Offline Goal-Conditioned Reinforcement Learning for Safety-Critical Tasks with Recovery Policycited by 5
  29. 2023Data-efficient Deep Reinforcement Learning for Vehicle Trajectory Controlcited by 4
  30. 2023BayRnTune: Adaptive Bayesian Domain Randomization via Strategic Fine-tuningcited by 3
  31. 2025Humanoid Agent via Embodied Chain-of-Action Reasoning with Multimodal Foundation Models for Zero-Shot Loco-Manipulationcited by 3
  32. 2026What Matters for Simulation to Online Reinforcement Learning on Real Robotscited by 3
  33. 2022Efficient Learning of Locomotion Skills through the Discovery of Diverse Environmental Trajectory Generator Priorscited by 2
  34. 2023Machine Learning Meets Advanced Robotic Manipulationcited by 2
  35. 2024An Efficient Model-Based Approach on Learning Agile Motor Skills without Reinforcementcited by 2
  36. 2024LoopSR: Looping Sim-and-Real for Lifelong Policy Adaptation of Legged Robotscited by 2
  37. 2025A Multi-Fidelity Control Variate Approach for Policy Gradient Estimationcited by 2
  38. 2025Transferable Latent-to-Latent Locomotion Policy for Efficient and Versatile Motion Control of Diverse Legged Robotscited by 2

50 most cited of 71 All 71 (JSON)