mlsanityData to 7 Oct 2026

Grow Your Limits: Continuous Improvement with Real-World RL for Robotic Locomotion

arXiv 2310.17634 ↗First posted 26 Oct 2023cs.RO cs.AI cs.LGcited by 21

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning (RL) can enable robots to autonomously acquire complex behaviors, such as legged locomotion. However, RL in the real world is complicated by constraints on efficiency, safety, and overall training stability, which limits its practical applicability. We present APRL, a policy regularization framework that modulates the robot's exploration over the course of training, striking a balance between flexible improvement potential and focused, efficient exploration. APRL enables a quadrupedal robot to efficiently learn to walk entirely in the real world within minutes and continue to improve with more training where prior work saturates in performance. We demonstrate that continued training with APRL results in a policy that is substantially more capable of navigating challenging situations and is able to adapt to changes in dynamics with continued training.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 40 of its 64 references that are in the map. First posted on arXiv 26 Oct 2023.

Builds on

40 of 64 references.

References · 40

  1. 2016Layer Normalizationcited by 5,579
  2. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  3. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  4. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  5. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 494
  6. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 430
  7. 2019Learning agile and dynamic motor skills for legged robotscited by 402
  8. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 395
  9. 2022DayDreamer: World Models for Physical Robot Learningcited by 372
  10. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 312
  11. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 278
  12. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 277
28 more
  1. 2016Deep Visual Foresight for Planning Robot Motioncited by 276
  2. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
  3. 2022The Primacy Bias in Deep Reinforcement Learningcited by 242
  4. 2020Learning Agile Robotic Locomotion Skills by Imitating Animalscited by 229
  5. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  6. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  7. 2019Deep Dynamics Models for Learning Dexterous Manipulationcited by 200
  8. 2020SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learningcited by 150
  9. 2022Rapid Locomotion via Reinforcement Learningcited by 130
  10. 2020The Ingredients of Real-World Robotic Reinforcement Learningcited by 111
  11. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 111
  12. 2021Dropout Q-Functions for Doubly Efficient Reinforcement Learningcited by 108
  13. 2018Model-Based Active Explorationcited by 105
  14. 2020Learning to Walk in the Real World with Minimal Human Effortcited by 103
  15. 2021Minimizing Energy Consumption Leads to the Emergence of Gaits in Legged Robotscited by 67
  16. 2019Data Efficient Reinforcement Learning for Legged Robotscited by 60
  17. 2019Sim-to-Real Transfer for Biped Locomotioncited by 50
  18. 2021Fast and Efficient Locomotion via Learned Gait Transitionscited by 50
  19. 2019Policies Modulating Trajectory Generatorscited by 47
  20. 2021Learning to Jump from Pixelscited by 39
  21. 2023Efficient Deep Reinforcement Learning Requires Regulating Overfittingcited by 36
  22. 2023Learning and Adapting Agile Locomotion Skills by Transferring Experiencecited by 25
  23. 2022Advanced Skills through Multiple Adversarial Motion Priors in Reinforcement Learningcited by 23
  24. 2020Representation-Free Model Predictive Control for Dynamic Motions in Quadrupedscited by 11
  25. 2021Cat-like Jumping and Landing of Legged Robots in Low-gravity Using Deep Reinforcement Learningcited by 11
  26. 2018Simulator Predictive Control: Using Learned Task Representations and MPC for Zero-Shot Generalization and Sequencingcited by 4
  27. 2017From the Lab to the Desert: Fast Prototyping and Learning of Robot Locomotioncited by 3
  28. 2022Multi-Modal Legged Locomotion Framework with Automated Residual Reinforcement Learningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 5; 2025: 9; 2026: 4.

21 papers.

Sort:

By area:

  1. 2025A Vision-Language-Action-Critic Model for Robotic Real-World Reinforcement Learningcited by 66
  2. 2024One Policy to Run Them All: an End-to-end Learning Approach to Multi-Embodiment Locomotioncited by 34
  3. 2023Uni-O4: Unifying Online and Offline Deep Reinforcement Learning with Multi-Step On-Policy Optimizationcited by 23
  4. 2024A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agentscited by 16
  5. 2025Towards Embodiment Scaling Laws in Robot Locomotioncited by 12
  6. 2025Gait in Eight: Efficient On-Robot Learning for Omnidirectional Quadruped Locomotioncited by 7
  7. 2024Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolutioncited by 6
  8. 2025Multistep Quasimetric Learning for Scalable Goal-conditioned Reinforcement Learningcited by 6
  9. 2023Conformal Policy Learning for Sensorimotor Control Under Distribution Shiftscited by 5
  10. 2026Simulation Distillation: Pretraining World Models in Simulation for Rapid Real-World Adaptationcited by 3
  11. 2025Optimizing Quantum Circuits via ZX Diagrams using Reinforcement Learning and Graph Neural Networkscited by 2
  12. 2023Efficient Sparse-Reward Goal-Conditioned Reinforcement Learning with a High Replay Ratio and Regularizationcited by 1
9 more
  1. 2024LoopSR: Looping Sim-and-Real for Lifelong Policy Adaptation of Legged Robotscited by 1
  2. 2025TIMRL: A Novel Meta-Reinforcement Learning Framework for Non-Stationary and Multi-Task Environmentscited by 1
  3. 2025Robust Localization, Mapping, and Navigation for Quadruped Robotscited by 1
  4. 2026Shape Your Body: Value Gradients for Multi-Embodiment Robot Designcited by 1
  5. 2024BricksRL: A Platform for Democratizing Robotics and Reinforcement Learning Research and Education with LEGOcited by 0
  6. 2025Dynamic Obstacle Avoidance with Bounded Rationality Adversarial Reinforcement Learningcited by 0
  7. 2025Expanding-and-Shrinking Binary Neural Networkscited by 0
  8. 2026SafeExplorer: An Unbiased Policy Gradient for Reinforcement Learning with Recovery Interventionscited by 0
  9. 2026HELP: Human-Efficient Large-Scale Robot Post-Training with Rollout Segmentationcited by 0