mlsanityData to 14 Oct 2026

Safe Online Learning via Smooth Safety-Structured Policy Composition

arXiv 2606.31320 ↗First posted 30 Jun 2026cs.LG cs.ROcited by 0

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Safe online reinforcement learning requires policies to respect safety constraints while maintaining smooth optimization dynamics. Existing approaches typically rely on either strict safety enforcement via action interventions, which introduce discontinuities in system interaction and learning, or soft safety constraint formulations, which preserve smooth learning but provide limited safety assurance. We propose AutoSafe, a safety-aware policy architecture that integrates structured safety monitoring and intervention directly into the action generation process. This design enables smooth, risk-dependent transitions between performance-driven and safety-preserving behaviors, resulting in continuous online interaction and learning dynamics. Empirical results across a suite of continuous-control benchmarks demonstrate strong safety enforcement without sacrificing learning smoothness. We further validate AutoSafe on a physical cart-pole system, highlighting its practical effectiveness for safe online learning in the real world.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 30 of its 47 references that are in the map. First posted on arXiv 30 Jun 2026.

Builds on

30 of 47 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Safe Reinforcement Learning using Action Projection: Safeguard the Policy or the Environment?16 Sep
  2. 2024Reinforcement Learning with Adaptive Regularization for Safe Control of Critical Systems23 Apr
  3. 2024Sampling-based Safe Reinforcement Learning for Nonlinear Dynamical Systems6 Mar

Other references · 27

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015Continuous control with deep reinforcement learningcited by 3,407
  4. 2021Convex optimizationcited by 2,515
  5. 2017Constrained Policy Optimizationcited by 983
  6. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 599
  7. 2019Control Barrier Functions: Theory and Applicationscited by 372
  8. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 289
  9. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 213
  10. 2020Learning to Walk in the Real World with Minimal Human Effortcited by 111
  11. 2023The Safety Filter: A Unified View of Safety-Critical Control in Autonomous Systemscited by 88
  12. 2021Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformerscited by 79
15 more
  1. 2021safe-control-gym: a Unified Benchmark Suite for Safe Learning-based Control and Reinforcement Learning in Roboticscited by 50
  2. 2018Linear model predictive safety certification for learning-based controlcited by 37
  3. 2019Control Regularization for Reduced Variance Reinforcement Learningcited by 37
  4. 2022Provably Safe Reinforcement Learning: Conceptual Analysis, Survey, and Benchmarkingcited by 34
  5. 2019Neural Simplex Architecturecited by 29
  6. 2022Lyapunov Design for Robust and Efficient Robotic Reinforcement Learningcited by 28
  7. 2020Deep Reinforcement Learning for Closed-Loop Blood Glucose Controlcited by 26
  8. 2021Safe Pontryagin Differentiable Programmingcited by 26
  9. 2020Multi-Layered Safety for Legged Robots via Control Barrier Functions and Model Predictive Controlcited by 25
  10. 2021Bayesian Controller Fusion: Leveraging Control Priors in Deep Reinforcement Learning for Roboticscited by 20
  11. 2018Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learningcited by 17
  12. 2021FaSTrack: a Modular Framework for Real-Time Motion Planning and Guaranteed Safe Trackingcited by 15
  13. 2023Physics-Regulated Deep Reinforcement Learning: Invariant Embeddingscited by 10
  14. 2022Cloud-Edge Training Architecture for Sim-to-Real Deep Reinforcement Learningcited by 5
  15. 2025The Use of the Simplex Architecture to Enhance Safety in Deep-Learning-Powered Autonomous Systemscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Safe-RULE: Safe Reinforcement UnLEarning
  2. 2026LexiSafe: Offline Safe Reinforcement Learning with Lexicographic Safety-Reward Hierarchy
  3. 2025CBF-RL: Safety Filtering Reinforcement Learning in Training with Control Barrier Functions
  4. 2024Safety through Permissibility: Shield Construction for Fast and Safe Reinforcement Learning
  5. 2026Robust Probabilistic Shielding for Safe Offline Reinforcement Learning
  6. 2025Safe and Near-Optimal Control with Online Dynamics Learning
  7. 2025Provably Optimal Reinforcement Learning under Safety Filtering
  8. 2024Towards Fast Safe Online Reinforcement Learning via Policy Finetuning