mlsanityData to 14 Oct 2026

Physics-model-guided Worst-case Sampling for Safe Reinforcement Learning

arXiv 2412.13224 ↗First posted 17 Dec 2024cs.RO cs.AI cs.LGcited by 0

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Real-world accidents in learning-enabled CPS frequently occur in challenging corner cases. During the training of deep reinforcement learning (DRL) policy, the standard setup for training conditions is either fixed at a single initial condition or uniformly sampled from the admissible state space. This setup often overlooks the challenging but safety-critical corner cases. To bridge this gap, this paper proposes a physics-model-guided worst-case sampling strategy for training safe policies that can handle safety-critical cases toward guaranteed safety. Furthermore, we integrate the proposed worst-case sampling strategy into the physics-regulated deep reinforcement learning (Phy-DRL) framework to build a more data-efficient and safe learning algorithm for safety-critical CPS. We validate the proposed training strategy with Phy-DRL through extensive experiments on a simulated cart-pole system, a 2D quadrotor, a simulated and a real quadruped robot, showing remarkably improved sampling efficiency to learn more robust safe policies.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 24 of its 49 references that are in the map. First posted on arXiv 17 Dec 2024.

Builds on

24 of 49 references.

References · 24

  1. 2015Continuous control with deep reinforcement learningcited by 3,407
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  3. 2017Constrained Policy Optimizationcited by 983
  4. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  5. 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 744
  6. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 289
  7. 2018Learning to Drive in a Daycited by 273
  8. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 213
  9. 2019Active Domain Randomizationcited by 170
  10. 2020Safe Reinforcement Learning in Constrained Markov Decision Processescited by 115
  11. 2021Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformerscited by 79
  12. 2021Robot Learning from Randomized Simulations: A Reviewcited by 63
12 more
  1. 2021safe-control-gym: a Unified Benchmark Suite for Safe Learning-based Control and Reinforcement Learning in Roboticscited by 50
  2. 2021Stabilizing Neural Control Using Self-Learned Almost Lyapunov Criticscited by 35
  3. 2021Description of Corner Cases in Automated Driving: Goals and Challengescited by 35
  4. 2022Lyapunov Design for Robust and Efficient Robotic Reinforcement Learningcited by 28
  5. 2018Verification for Machine Learning, Autonomy, and Neural Networks Surveycited by 24
  6. 2021Bayesian Controller Fusion: Leveraging Control Priors in Deep Reinforcement Learning for Roboticscited by 20
  7. 2023Stable and Safe Reinforcement Learning via a Barrier-Lyapunov Actor-Critic Approachcited by 13
  8. 2023Physics-Regulated Deep Reinforcement Learning: Invariant Embeddingscited by 10
  9. 2020Safety Verification of Neural Network Controlled Systemscited by 9
  10. 2022Equipping Black-Box Policies with Model-Based Advice for Stable Nonlinear Controlcited by 6
  11. 2024Simplex-enabled Safe Continual Learning Machinecited by 2
  12. 2019Synthesis of Admissible Shieldscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Sampling-Based Safe Reinforcement Learning
  2. 2024Sampling-based Safe Reinforcement Learning for Nonlinear Dynamical Systems
  3. 2026Safe Reinforcement Learning using Ideas from Model Predictive Control
  4. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applications
  5. 2026Dyna-Style Safety Augmented Reinforcement Learning: Staying Safe in the Face of Uncertainty
  6. 2024Safety-Driven Deep Reinforcement Learning Framework for Cobots: A Sim2Real Approach
  7. 2024Physics-informed RL for Maximal Safety Probability Estimation
  8. 2025Real-DRL: Teach and Learn in Reality