mlsanityData to 14 Oct 2026

Adaptive Shielding for Safe Reinforcement Learning under Hidden-Parameter Dynamics Shifts

arXiv 2506.11033 ↗First posted 20 May 2025cs.LG cs.AIcited by 2

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Unseen shifts in environment dynamics, driven by hidden parameters such as friction or gravity, create a challenge for maintaining safety. We address this challenge by proposing Adaptive Shielding, a framework for safe reinforcement learning in constrained hidden-parameter Markov decision processes. A function encoder infers a low-dimensional representation of the underlying dynamics online from transition data, allowing the shield to adapt. To ensure safety during this process, we use a two-layer strategy. First, we introduce safety-regularized optimization that proactively trains the policy away from high-cost regions. Second, the adaptive shielding reactively uses the inferred dynamics to forecast safety risks and applies uncertainty-aware bounds using conformal prediction to filter unsafe actions. We prove that prediction errors in the shielding connect with bounds on the average cost rate. Empirically, across Safe-Gym benchmarks with varying hidden parameters, our approach outperforms baselines on the return-safety trade-off and generalizes reliably to unseen dynamics, while incurring only modest execution-time overhead. Code is available at https://github.com/safe-autonomy-lab/AdaptiveShieldingFE.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 42 of its 58 references that are in the map. First posted on arXiv 20 May 2025.

Builds on

42 of 58 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024ActSafe: Active Exploration with Safety Constraints for Reinforcement Learning12 Oct
  2. 2024Enhancing Efficiency of Safe Reinforcement Learning via Sample Manipulation31 May

Other references · 40

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  3. 2015Trust Region Policy Optimizationcited by 2,580
  4. 2017Constrained Policy Optimizationcited by 983
  5. 2019Conformal Prediction Under Covariate Shiftcited by 519
  6. 2017Safe Reinforcement Learning via Shieldingcited by 449
  7. 2018Reward Constrained Policy Optimizationcited by 417
  8. 2021Adaptive Conformal Inference Under Distribution Shiftcited by 375
  9. 2020Projection-Based Constrained Policy Optimizationcited by 203
  10. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 170
  11. 2023Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmarkcited by 143
  12. 2022Conformal Inference for Online Prediction with Arbitrary Distribution Shiftscited by 128
28 more
  1. 2022Safe Planning in Dynamic Environments using Conformal Predictioncited by 119
  2. 2020Safe Reinforcement Learning in Constrained Markov Decision Processescited by 115
  3. 2021Learning Policies with Zero or Bounded Constraint Violation for Constrained MDPscited by 80
  4. 2023OmniSafe: An Infrastructure for Accelerating Safe Reinforcement Learning Researchcited by 80
  5. 2022Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentationcited by 69
  6. 2022Enforcing Hard Constraints with Soft Barriers: Safe Reinforcement Learning in Unknown Stochastic Environmentscited by 60
  7. 2022Constrained Update Projection Approach to Safe Policy Optimizationcited by 58
  8. 2022Safe Reinforcement Learning via Shielding under Partial Observabilitycited by 56
  9. 2022Contextualize Me -- The Case for Context in Reinforcement Learningcited by 46
  10. 2023Safe Reinforcement Learning via Probabilistic Logic Shieldscited by 42
  11. 2023Iterative Reachability Estimation for Safe Reinforcement Learningcited by 37
  12. 2021Context-Aware Safe Reinforcement Learning for Non-Stationary Environmentscited by 33
  13. 2022Evaluating Model-free Reinforcement Learning toward Safety-critical Taskscited by 22
  14. 2022Achieving Zero Constraint Violation for Constrained Reinforcement Learning via Conservative Natural Policy Gradient Primal-Dual Algorithmcited by 20
  15. 2023Dynamics Generalisation in Reinforcement Learning via Adaptive Context-Aware Policiescited by 20
  16. 2022Hypernetworks for Zero-shot Transfer in Reinforcement Learningcited by 19
  17. 2019Single Episode Policy Transfer in Reinforcement Learningcited by 18
  18. 2024Zero-Shot Reinforcement Learning via Function Encoderscited by 18
  19. 2023AutoCost: Evolving Intrinsic Cost for Zero-violation Reinforcement Learningcited by 17
  20. 2024Dreaming of Many Worlds: Learning Contextual World Models Aids Zero-Shot Generalizationcited by 16
  21. 2022An Adaptive Deep RL Method for Non-Stationary Environments with Piecewise Stable Contextcited by 15
  22. 2024Acquiring Diverse Skills using Curriculum Reinforcement Learning with Mixture of Expertscited by 14
  23. 2024A CMDP-within-online framework for Meta-Safe Reinforcement Learningcited by 12
  24. 2022Safe and Efficient Reinforcement Learning Using Disturbance-Observer-Based Control Barrier Functionscited by 11
  25. 2024Zero-Shot Transfer of Neural ODEscited by 10
  26. 2025Function Encoders: A Principled Approach to Transfer Learning in Hilbert Spacescited by 10
  27. 2023Safe POMDP Online Planning via Shieldingcited by 8
  28. 2024Safe POMDP Online Planning among Dynamic Agents via Adaptive Conformal Predictioncited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Reliability-Asymmetric Spacecraft Autonomy: Co-Designing a Capable Learned GNC Stack with a Verified, Adaptation-Aware Runtime Shieldcited by 0
  2. 2026Conformal Recovery-Deadline Certificates for Runtime Assurance of Adapting Controllerscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Adaptive Probabilistic Shielding by Learning MDPs for Safe Reinforcement Learning
  2. 2024Safe Reinforcement Learning in Black-Box Environments via Adaptive Shielding
  3. 2024Leveraging Approximate Model-based Shielding for Probabilistic Safety Guarantees in Continuous Environments
  4. 2026Robust Shielding for Safe Reinforcement Learning
  5. 2025Probabilistic Shielding for Safe Reinforcement Learning
  6. 2025ProSh: Probabilistic Shielding for Model-free Reinforcement Learning
  7. 2026Safe Reinforcement Learning via Recovery-based Shielding with Gaussian Process Dynamics Models
  8. 2026CAPSULE: Control-Theoretic Action Perturbations for Safe Uncertainty-Aware Reinforcement Learning