mlsanityData to 14 Oct 2026

SteinGate: Tail-Sensitive Safe Reinforcement Learning via Stein Discrepancy

arXiv 2607.13175 ↗First posted 14 Jul 2026cs.LG cs.AIcited by 1

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Safe reinforcement learning typically enforces safety by bounding expected cumulative costs, a criterion that often fails to detect rare but catastrophic tail events. To overcome these limitations, this paper introduces SteinGate, a boundary-aware distributional safety certificate that replaces fragile tail fitting with a robust consistency check using Kernelized Stein Discrepancy while accounting for boundary atoms induced by clipped costs. SteinGate evaluates whether observed policy rollout costs remain consistent with a safe reference distribution, providing a non-parametric safety certificate. This certificate is used to dynamically adapt the learning regime: favoring reward-improving policy updates when rollouts remain consistent with the safe reference and switching to recovery behavior when the cost tail deviates. Experiments on continuous-control benchmarks demonstrate that SteinGate significantly reduces both the frequency and severity of constraint violations during training while maintaining competitive returns relative to state-of-the-art baselines.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 28 of its 40 references that are in the map. First posted on arXiv 14 Jul 2026.

Builds on

28 of 40 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026Extreme Value Policy Optimization for Safe Reinforcement Learning17 Jan

Other references · 27

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Trust Region Policy Optimizationcited by 2,580
  3. 2017Constrained Policy Optimizationcited by 983
  4. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  5. 2018Reward Constrained Policy Optimizationcited by 417
  6. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 337
  7. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 316
  8. 2019Reinforcement Learning in Healthcare: A Surveycited by 310
  9. 2016A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluationcited by 236
  10. 2020Projection-Based Constrained Policy Optimizationcited by 203
  11. 2015Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approachcited by 170
  12. 2016A Kernel Test of Goodness of Fitcited by 143
15 more
  1. 2013Risk-sensitive Reinforcement Learningcited by 130
  2. 2015Measuring Sample Quality with Stein's Methodcited by 94
  3. 2023OmniSafe: An Infrastructure for Accelerating Safe Reinforcement Learning Researchcited by 80
  4. 2022Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentationcited by 69
  5. 2024Safe Offline Reinforcement Learning with Feasibility-Guided Diffusion Modelcited by 64
  6. 2022Enforcing Hard Constraints with Soft Barriers: Safe Reinforcement Learning in Unknown Stochastic Environmentscited by 60
  7. 2022Constrained Update Projection Approach to Safe Policy Optimizationcited by 58
  8. 2020Cautious Adaptation For Reinforcement Learning in Safety-Critical Settingscited by 42
  9. 2024Constraint-Adaptive Policy Switching for Offline Safe Reinforcement Learningcited by 13
  10. 2022Quantile Constrained Reinforcement Learning: A Reinforcement Learning Framework Constraining Outage Probabilitycited by 9
  11. 2022Posterior Coreset Construction with Kernelized Stein Discrepancy for Model-Based Reinforcement Learningcited by 7
  12. 2023On-Policy Policy Gradient Reinforcement Learning Without On-Policy Samplingcited by 6
  13. 2023Extreme Risk Mitigation in Reinforcement Learning using Extreme Value Theorycited by 5
  14. 2020A Sigmoidal Approximation for Chance-Constrained Nonlinear Programscited by 4
  15. 2025Online Optimization for Offline Safe Reinforcement Learningcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026BCPPO: Bachelier-Inspired Constrained Proximal Policy Optimization for Tail-Risk-Aware Safe Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Provably Safe, Yet Scalable Reinforcement Learning
  2. 2024Safe Reinforcement Learning with Learned Non-Markovian Safety Constraints
  3. 2024Revisiting Safe Exploration in Safe Reinforcement learning
  4. 2026Safe on Average, Unsafe in the Tail: When Is the Episodic-Cost Tail Controllable?
  5. 2025SB-TRPO: Towards Safe Reinforcement Learning with Hard Constraints
  6. 2026Evaluation Metrics for Safe Reinforcement Learning
  7. 2025Safety Modulation: Enhancing Safety in Reinforcement Learning through Cost-Modulated Rewards
  8. 2025Provably Optimal Reinforcement Learning under Safety Filtering