mlsanityData to 14 Oct 2026

ResSafe: Learning Safety Filtering with Residual Reinforcement Learning for Humanoids

arXiv 2609.15988 ↗First posted 14 Sep 2026cs.ROcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Safe control of humanoid robots remains challenging due to their high-dimensional dynamics, contact-rich interactions, and sensitivity to disturbances. Although reinforcement learning has enabled effective locomotion and motion tracking, learned policies can still generate unsafe actions that lead to instability or falls. In this work, we propose residual reinforcement learning as an implicit safety-filtering mechanism for safe humanoid control. Instead of relying on a single nominal policy to simultaneously balance performance, safety, and robustness, we decouple performance and safety. The nominal policy focuses solely on task performance, while a residual policy learns safety corrections. This decoupling leads to a better performance--safety Pareto trade-off and avoids the need for careful tuning of multiple competing reward terms within a single policy training. We show that the residual policy can act as an implicit safety filter.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 46 of its 55 references that are in the map. First posted on arXiv 14 Sep 2026.

Builds on

46 of 55 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026ContractionPPO: Certified Reinforcement Learning via Differentiable Contraction Layers20 Mar
  2. 2025CLF-RL: Control Lyapunov Function Guided Reinforcement Learning12 Aug
  3. 2024Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control30 Jan

Other references · 43

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  3. 2017Constrained Policy Optimizationcited by 983
  4. 2019Control Barrier Functions: Theory and Applicationscited by 372
  5. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 297
  6. 2016Control Barrier Function Based Quadratic Programs for Safety Critical Systemscited by 287
  7. 2018Residual Reinforcement Learning for Robot Controlcited by 274
  8. 2017Asymmetric Actor Critic for Image-Based Robot Learningcited by 238
  9. 2017Hamilton-Jacobi Reachability: A Brief Overview and Recent Advancescited by 201
  10. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 175
  11. 2017A General Safety Framework for Learning-Based Control in Uncertain Robotic Systemscited by 163
  12. 2018A predictive safety filter for learning-based control of constrained nonlinear dynamical systemscited by 118
31 more
  1. 2020Learning to be Safe: Deep RL with a Safety Criticcited by 116
  2. 2023The Safety Filter: A Unified View of Safety-Critical Control in Autonomous Systemscited by 88
  3. 2024Agile But Safe: Learning Collision-Free High-Speed Legged Locomotioncited by 73
  4. 2020Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functionscited by 68
  5. 2025ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learningcited by 66
  6. 2021Safe Reinforcement Learning Using Advantage-Based Interventioncited by 47
  7. 2023How to Train Your Neural Control Barrier Function: Learning Safety Filters for Complex Input-Constrained Systemscited by 44
  8. 2025ResMimic: From General Motion Tracking to Humanoid Whole-body Loco-Manipulation via Residual Learningcited by 41
  9. 2022ISAACS: Iterative Soft Adversarial Actor-Critic for Safetycited by 28
  10. 2025HuB: Learning Extreme Humanoid Balancecited by 28
  11. 2025VIRAL: Visual Sim-to-Real at Scale for Humanoid Loco-Manipulationcited by 25
  12. 2025CBF-RL: Safety Filtering Reinforcement Learning in Training with Control Barrier Functionscited by 17
  13. 2024Gameplay Filters: Robust Zero-Shot Safety through Adversarial Imaginationcited by 16
  14. 2020Safe reinforcement learning for probabilistic reachability and safety specifications: A Lyapunov-based approachcited by 13
  15. 2024Safety Filters for Black-Box Dynamical Systems by Learning Discriminating Hyperplanescited by 12
  16. 2025Agility Meets Stability: Versatile Humanoid Control with Heterogeneous Datacited by 11
  17. 2024Efficient Residual Learning with Mixture-of-Experts for Universal Dexterous Graspingcited by 10
  18. 2025Safety with Agency: Human-Centered Safety Filter with Application to AI-Assisted Motorsportscited by 8
  19. 2025SHIELD: Safety on Humanoids via CBFs In Expectation on Learned Dynamicscited by 7
  20. 2022Humanoid Self-Collision Avoidance Using Whole-Body Control with Control Barrier Functionscited by 5
  21. 2024Certifiable Reachability Learning Using a New Lipschitz Continuous Value Functioncited by 5
  22. 2025Verifiable Safety Q-Filters via Hamilton-Jacobi Reachability and Multiplicative Q-Networkscited by 5
  23. 2024ProxFly: Robust Control for Close Proximity Quadcopter Flight via Residual Reinforcement Learningcited by 4
  24. 2026Bellman Value Decomposition for Task Logic in Safe Optimal Controlcited by 4
  25. 2024Cooptimizing Safety and Performance with a Control-Constrained Formulationcited by 3
  26. 2024Safe Control of Quadruped in Varying Dynamics via Safety Index Adaptationcited by 3
  27. 2025Whole-Body Bilateral Teleoperation with Multi-Stage Object Parameter Estimation for Wheeled Humanoid Locomanipulationcited by 3
  28. 2026Deep QP Safety Filter: Model-free Learning for Reachability-based Safety Filtercited by 2
  29. 2023Improving Wind Resistance Performance of Cascaded PID Controlled Quadcopters using Residual Reinforcement Learningcited by 1
  30. 2025Servo-Controllers with Operational Constraintscited by 1
  31. 2026Safety-Critical Whole-Body Control for Humanoid Robots via Input-to-State Safe Control Barrier Functionscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026FAITH: Feasibility-Aware Safety-Filtered RL for High-Dimensional Systems
  2. 2026Shield-Loco: Shielding Locomotion Policies with Predictive Safety Filtering
  3. 2026Safe Reinforcement Learning with Receding-Horizon Safety Critic and Differentiable Action Shields
  4. 2025SafeFall: Learning Protective Control for Humanoid Robots
  5. 2025RuN: Residual Policy for Natural Humanoid Locomotion
  6. 2026Towards Bridging the Gap between Large-Scale Pretraining and Efficient Finetuning for Humanoid Control
  7. 2026Model-Informed Safe Reinforcement Learning for Bipedal Locomotion via Step-to-Step Prediction
  8. 2025Robot Trains Robot: Automatic Real-World Policy Adaptation and Learning for Humanoids