mlsanityData to 14 Oct 2026

A Safety Modulator Actor-Critic Method in Model-Free Safe Reinforcement Learning and Application in UAV Hovering

arXiv 2410.06847 ↗First posted 9 Oct 2024cs.AI cs.LG cs.ROcited by 0

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

This paper proposes a safety modulator actor-critic (SMAC) method to address safety constraint and overestimation mitigation in model-free safe reinforcement learning (RL). A safety modulator is developed to satisfy safety constraints by modulating actions, allowing the policy to ignore safety constraint and focus on maximizing reward. Additionally, a distributional critic with a theoretical update rule for SMAC is proposed to mitigate the overestimation of Q-values with safety constraints. Both simulation and real-world scenarios experiments on Unmanned Aerial Vehicles (UAVs) hovering confirm that the SMAC can effectively maintain safety constraints and outperform mainstream baseline algorithms.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 15 of its 31 references that are in the map. First posted on arXiv 9 Oct 2024.

Builds on

15 of 31 references.

References · 15

  1. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  2. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  3. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  4. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  5. 2018Reward Constrained Policy Optimizationcited by 417
  6. 2020dm_control: Software and Tasks for Continuous Controlcited by 375
  7. 2016Safe and Efficient Off-Policy Reinforcement Learningcited by 264
  8. 2018Safe Exploration in Continuous Action Spacescited by 247
  9. 2018Distributed Distributional Deterministic Policy Gradientscited by 199
  10. 2020Safe Reinforcement Learning in Constrained Markov Decision Processescited by 115
  11. 2020Learning to Walk in the Real World with Minimal Human Effortcited by 111
  12. 2020Distributional Soft Actor-Critic: Off-Policy Reinforcement Learning for Addressing Value Estimation Errorscited by 85
3 more
  1. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 58
  2. 2023State-wise Safe Reinforcement Learning: A Surveycited by 55
  3. 2022Towards Safe Reinforcement Learning with a Safety Editor Policycited by 30

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Safety Modulation: Enhancing Safety in Reinforcement Learning through Cost-Modulated Rewards
  2. 2026Towards Safe Reinforcement Learning with Reduced Conservativeness: A Case Study on Drone Flight Control
  3. 2026Learning over Forward-Invariant Policy Classes: Reinforcement Learning without Safety Concerns
  4. 2024Safety Filtering While Training: Improving the Performance and Sample Efficiency of Reinforcement Learning Agents
  5. 2024Sampling-based Safe Reinforcement Learning for Nonlinear Dynamical Systems
  6. 2026SafePG: Safe and Globally Optimal Reinforcement Learning with Hard Constraints
  7. 2024Safety Optimized Reinforcement Learning via Multi-Objective Policy Optimization
  8. 2025Safe But Not Sorry: Reducing Over-Conservatism in Safety Critics via Uncertainty-Aware Modulation