mlsanityData to 7 Oct 2026

Simplex-enabled Safe Continual Learning Machine

arXiv 2409.05898 ↗First posted 5 Sep 2024cs.LG cs.AI cs.ROcited by 2

Not on the map. Why?

Abstract (arXiv)

This paper proposes the SeC-Learning Machine: Simplex-enabled safe continual learning for safety-critical autonomous systems. The SeC-learning machine is built on Simplex logic (that is, ``using simplicity to control complexity'') and physics-regulated deep reinforcement learning (Phy-DRL). The SeC-learning machine thus constitutes HP (high performance)-Student, HA (high assurance)-Teacher, and Coordinator. Specifically, the HP-Student is a pre-trained high-performance but not fully verified Phy-DRL, continuing to learn in a real plant to tune the action policy to be safe. In contrast, the HA-Teacher is a mission-reduced, physics-model-based, and verified design. As a complementary, HA-Teacher has two missions: backing up safety and correcting unsafe learning. The Coordinator triggers the interaction and the switch between HP-Student and HA-Teacher. Powered by the three interactive components, the SeC-learning machine can i) assure lifetime safety (i.e., safety guarantee in any continual-learning stage, regardless of HP-Student's success or convergence), ii) address the Sim2Real gap, and iii) learn to tolerate unknown unknowns in real plants. The experiments on a cart-pole system and a real quadruped robot demonstrate the distinguished features of the SeC-learning machine, compared with continual learning built on state-of-the-art safe DRL frameworks with approaches to addressing the Sim2Real gap.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 33 of its 53 references that are in the map. First posted on arXiv 5 Sep 2024.

Builds on

33 of 53 references.

References · 33

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  5. 2017Hindsight Experience Replaycited by 1,096
  6. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  7. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  8. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  9. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 347
  10. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 277
  11. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  12. 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 268
21 more
  1. 2018Learning to Drive in a Daycited by 266
  2. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 256
  3. 2018Residual Reinforcement Learning for Robot Controlcited by 242
  4. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 227
  5. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  6. 2017Preparing for the Unknown: Learning a Universal Policy with Online System Identificationcited by 129
  7. 2021Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformerscited by 71
  8. 2021Auto-Tuned Sim-to-Real Transfercited by 45
  9. 2019Control Regularization for Reduced Variance Reinforcement Learningcited by 34
  10. 2021Stabilizing Neural Control Using Self-Learned Almost Lyapunov Criticscited by 33
  11. 2022Safe Reinforcement Learning for Legged Locomotioncited by 29
  12. 2019How to pick the domain randomization parameters for sim-to-real transfer of reinforcement learning policies?cited by 25
  13. 2020Guided Constrained Policy Optimization for Dynamic Quadrupedal Robot Locomotioncited by 24
  14. 2022Lyapunov Design for Robust and Efficient Robotic Reinforcement Learningcited by 20
  15. 2021Bayesian Controller Fusion: Leveraging Control Priors in Deep Reinforcement Learning for Roboticscited by 18
  16. 2021Vision-Guided Quadrupedal Locomotion in the Wild with Multi-Modal Delay Randomizationcited by 18
  17. 2023Stable and Safe Reinforcement Learning via a Barrier-Lyapunov Actor-Critic Approachcited by 11
  18. 2023Physics-Regulated Deep Reinforcement Learning: Invariant Embeddingscited by 8
  19. 2020SL1-Simplex: Safe Velocity Regulation of Self-Driving Vehicles in Dynamic and Unforeseen Environmentscited by 7
  20. 2022Equipping Black-Box Policies with Model-Based Advice for Stable Nonlinear Controlcited by 5
  21. 2022Cloud-Edge Training Architecture for Sim-to-Real Deep Reinforcement Learningcited by 4

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2025Safe Continual Domain Adaptation after Sim2Real Transfer of Reinforcement Learning Policies in Roboticscited by 1
  2. 2024Physics-model-guided Worst-case Sampling for Safe Reinforcement Learningcited by 0