mlsanityData to 14 Oct 2026

Constraint-Aware Reinforcement Learning via Adaptive Action Scaling

arXiv 2510.11491 ↗First posted 13 Oct 2025cs.RO cs.LG cs.SYcited by 0

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Safe reinforcement learning (RL) seeks to mitigate unsafe behaviors that arise from exploration during training by reducing constraint violations while maintaining task performance. Existing approaches typically rely on a single policy to jointly optimize reward and safety, which can cause instability due to conflicting objectives, or they use external safety filters that override actions and require prior system knowledge. In this paper, we propose a modular cost-aware regulator that scales the agent's actions based on predicted constraint violations, preserving exploration through smooth action modulation rather than overriding the policy. The regulator is trained to minimize constraint violations while avoiding degenerate suppression of actions. Our approach integrates seamlessly with off-policy RL methods such as SAC and TD3, and achieves state-of-the-art return-to-cost ratios on Safety Gym locomotion tasks with sparse costs, reducing constraint violations by up to 126 times while increasing returns by over an order of magnitude compared to prior methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 29 of its 40 references that are in the map. First posted on arXiv 13 Oct 2025.

Builds on

29 of 40 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024A Dynamic Safety Shield for Safe and Efficient Reinforcement Learning of Navigation Tasks5 Dec
  2. 2024Dynamic Model Predictive Shielding for Provably Safe Reinforcement Learning22 May
  3. 2024Reinforcement Learning with Adaptive Regularization for Safe Control of Critical Systems23 Apr
  4. 2024Leveraging Approximate Model-based Shielding for Probabilistic Safety Guarantees in Continuous Environments1 Feb

Other references · 25

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  3. 2016Concrete Problems in AI Safetycited by 1,642
  4. 2017Constrained Policy Optimizationcited by 983
  5. 2018Reward Constrained Policy Optimizationcited by 417
  6. 2022DayDreamer: World Models for Physical Robot Learningcited by 400
  7. 2019Control Barrier Functions: Theory and Applicationscited by 372
  8. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 353
  9. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 316
  10. 2018Safe Exploration in Continuous Action Spacescited by 247
  11. 2024Reinforcement learningcited by 247
  12. 2022Multi-Task Learning as a Bargaining Gamecited by 215
13 more
  1. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 175
  2. 2023Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmarkcited by 143
  3. 2022Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentationcited by 69
  4. 2023Iterative Reachability Estimation for Safe Reinforcement Learningcited by 37
  5. 2022Towards Safe Reinforcement Learning with a Safety Editor Policycited by 30
  6. 2022Safe Reinforcement Learning Using Black-Box Reachability Analysiscited by 26
  7. 2023Constraint-Conditioned Policy Optimization for Versatile Safe Reinforcement Learningcited by 18
  8. 2022Spatial-Temporal-Aware Safe Multi-Agent Reinforcement Learning of Connected Autonomous Vehicles in Challenging Scenarioscited by 13
  9. 2023Safe Multi-Agent Reinforcement Learning for Behavior-Based Cooperative Navigationcited by 6
  10. 2021Safe Deep Reinforcement Learning for Multi-Agent Systems with Continuous Action Spacescited by 5
  11. 2023Safe Navigation and Obstacle Avoidance Using Differentiable Optimization Based Control Barrier Functionscited by 5
  12. 2023Safe Reinforcement Learning with Dead-Ends Avoidance and Recoverycited by 5
  13. 2024Spectral-Risk Safe Reinforcement Learning with Convergence Guaranteescited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Safe But Not Sorry: Reducing Over-Conservatism in Safety Critics via Uncertainty-Aware Modulation
  2. 2025Safety Modulation: Enhancing Safety in Reinforcement Learning through Cost-Modulated Rewards
  3. 2021Safe Continuous Control with Constrained Model-Based Policy Optimization
  4. 2026CSPO: Constraint-Sensitive Policy Optimization for Safe Reinforcement Learning
  5. 2026CAPSULE: Control-Theoretic Action Perturbations for Safe Uncertainty-Aware Reinforcement Learning
  6. 2024A Survey of Constraint Formulations in Safe Reinforcement Learning
  7. 2026From Cumulative Constraints to Adaptive Runtime Safety Control for Nonstationary Reinforcement Learning
  8. 2025Incentivizing Safer Actions in Policy Optimization for Constrained Reinforcement Learning