mlsanityData to 7 Oct 2026

Safe Reinforcement Learning for Legged Locomotion

arXiv 2203.02638 ↗First posted 5 Mar 2022cs.RO cs.AI cs.LGcited by 29

Not on the map. Why?

Abstract (arXiv)

Designing control policies for legged locomotion is complex due to the under-actuated and non-continuous robot dynamics. Model-free reinforcement learning provides promising tools to tackle this challenge. However, a major bottleneck of applying model-free reinforcement learning in real world is safety. In this paper, we propose a safe reinforcement learning framework that switches between a safe recovery policy that prevents the robot from entering unsafe states, and a learner policy that is optimized to complete the task. The safe recovery policy takes over the control when the learner policy violates safety constraints, and hands over the control back when there are no future safety violations. We design the safe recovery policy so that it ensures safety of legged locomotion while minimally intervening in the learning process. Furthermore, we theoretically analyze the proposed framework and provide an upper bound on the task performance. We verify the proposed framework in four locomotion tasks on a simulated and real quadrupedal robot: efficient gait, catwalk, two-leg balance, and pacing. On average, our method achieves 48.6% fewer falls and comparable or better rewards than the baseline methods in simulation. When deployed it on real-world quadruped robot, our training pipeline enables 34% improvement in energy efficiency for the efficient gait, 40.9% narrower of the feet placement in the catwalk, and two times more jumping duration in the two-leg balance. Our method achieves less than five falls over the duration of 115 minutes of hardware time.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 29 of its 45 references that are in the map. First posted on arXiv 5 Mar 2022.

Builds on

29 of 45 references.

References · 29

  1. 2015Trust Region Policy Optimizationcited by 2,503
  2. 2017Constrained Policy Optimizationcited by 928
  3. 2019Solving Rubik's Cube with a Robot Handcited by 699
  4. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 494
  5. 2018Reward Constrained Policy Optimizationcited by 397
  6. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 347
  7. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 302
  8. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 278
  9. 2018Safe Exploration in Continuous Action Spacescited by 237
  10. 2020Learning Agile Robotic Locomotion Skills by Imitating Animalscited by 229
  11. 2020Projection-Based Constrained Policy Optimizationcited by 195
  12. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 156
17 more
  1. 2018Learning to Walk via Deep Reinforcement Learningcited by 153
  2. 2017A General Safety Framework for Learning-Based Control in Uncertain Robotic Systemscited by 150
  3. 2019Lyapunov-based Safe Policy Optimization for Continuous Controlcited by 140
  4. 2018Learning-based Model Predictive Control for Safe Explorationcited by 116
  5. 2020Learning to be Safe: Deep RL with a Safety Criticcited by 109
  6. 2019Benchmarking Batch Deep Reinforcement Learning Algorithmscited by 108
  7. 2020Conservative Safety Critics for Explorationcited by 107
  8. 2020Learning to Walk in the Real World with Minimal Human Effortcited by 103
  9. 2018Neural Lander: Stable Drone Landing Control using Learned Dynamicscited by 82
  10. 2019Sim-to-Real Transfer for Biped Locomotioncited by 50
  11. 2019Learning Fast Adaptation with Meta Strategy Optimizationcited by 46
  12. 2020Constrained Upper Confidence Reinforcement Learningcited by 46
  13. 2020Cautious Reinforcement Learning with Logical Constraintscited by 44
  14. 2017Safe Learning of Quadrotor Dynamics Using Barrier Certificatescited by 32
  15. 2015Safety-Constrained Reinforcement Learning for MDPscited by 31
  16. 2022Quadruped Capturability and Push Recovery via a Switched-Systems Characterization of Dynamic Balancecited by 4
  17. 2020Protective Policy Transfercited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 6; 2024: 8; 2025: 4; 2026: 5.

29 papers.

Sort:

By area:

  1. 2022DayDreamer: World Models for Physical Robot Learningcited by 372
  2. 2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learningcited by 76
  3. 2024Agile But Safe: Learning Collision-Free High-Speed Legged Locomotioncited by 53
  4. 2023Not Only Rewards But Also Constraints: Applications on Legged Robot Locomotioncited by 33
  5. 2023Learning and Adapting Agile Locomotion Skills by Transferring Experiencecited by 25
  6. 2022Evaluating Model-free Reinforcement Learning toward Safety-critical Taskscited by 18
  7. 2023Constraint-Conditioned Policy Optimization for Versatile Safe Reinforcement Learningcited by 17
  8. 2022Safe Reinforcement Learning of Dynamic High-Dimensional Robotic Tasks: Navigation, Manipulation, Interactioncited by 15
  9. 2024Learning to Walk from Three Minutes of Real-World Data with Semi-structured Dynamics Modelscited by 14
  10. 2023Robotic Table Tennis: A Case Study into a High Speed Learning Systemcited by 12
  11. 2024Gameplay Filters: Robust Zero-Shot Safety through Adversarial Imaginationcited by 12
  12. 2024CaT: Constraints as Terminations for Legged Locomotion Reinforcement Learningcited by 11
17 more
  1. 2024SoloParkour: Constrained Reinforcement Learning for Visual Locomotion from Privileged Experiencecited by 10
  2. 2025Four Principles for Physically Interpretable World Modelscited by 10
  3. 2023Gradient Shaping for Multi-Constraint Safe Reinforcement Learningcited by 7
  4. 2022SafeRL-Kit: Evaluating Efficient Reinforcement Learning Methods for Safe Autonomous Drivingcited by 6
  5. 2024Spectral-Risk Safe Reinforcement Learning with Convergence Guaranteescited by 5
  6. 2025SafeMimic: Towards Safe and Autonomous Human-to-Robot Imitation for Mobile Manipulationcited by 5
  7. 2024Dexterous Legged Locomotion in Confined 3D Spaces with Reinforcement Learningcited by 4
  8. 2024Simplex-enabled Safe Continual Learning Machinecited by 2
  9. 2025Stability Enhancement in Reinforcement Learning via Adaptive Control Lyapunov Functioncited by 1
  10. 2026Constraint-Aware Diffusion Priors for High-Fidelity and Versatile Quadruped Locomotioncited by 1
  11. 2022Progress and summary of reinforcement learning on energy management of MPS-EVcited by 0
  12. 2023Efficient Exploration Using Extra Safety Budget in Constrained Policy Optimizationcited by 0
  13. 2025Real-DRL: Teach and Learn in Realitycited by 0
  14. 2026Reset-Free Reinforcement Learning for Real-World Agile Driving: An Empirical Studycited by 0
  15. 2026COP-Q: Safety-First Reinforcement Learning for Robot Control via Cholesky-Ordered Projectioncited by 0
  16. 2026SafeExplorer: An Unbiased Policy Gradient for Reinforcement Learning with Recovery Interventionscited by 0
  17. 2026Humanoid Safe Stop via Learned Stoppability Valuecited by 0