Reversible Simplex Supervision with Post-Action Debt Accounting for Goal-Reaching RL
On the map: Deep reinforcement learning methods → Safe and constrained RL
Abstract (arXiv)
Deploying reinforcement learning (RL) on multi-tonne robots calls for supervisory mechanisms that address both operational safety and progress toward task completion. However, repeated switching need not preserve task progress when a learned action increases storage before recovery takes control. We introduce reversible Simplex supervision with post-action debt accounting for a frozen finite-state policy and robust-adaptive recovery. Under exact sampled-state information and stated model and certificate conditions, we prove that recovery repayment exceeding a uniform triggering-edge debt bound guarantees finite switching and finite-sample goal entry. We formulate reachability-based certificate constructions for establishing these sufficient conditions. In 20 matched simulations, goal-entry counts are 20 with debt gating and 18 without it; the two remaining runs terminate under the supervisor's admissibility stopping rule. On an experimental 6000 kg robot, 24 asphalt and soft-terrain trials evaluate 50 ms supervision above a 1 kHz actuator stack; all eight triggered recoveries complete debt-gated re-entry. The experiments demonstrate the supervisory mechanism in the tested trials.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 21 of its 30 references that are in the map. First posted on arXiv 27 Jan 2026.
Builds on
21 of 30 references.
Closest ancestors · 2
Same direction, newest first
- 2024Safe Reinforcement Learning of Robot Trajectories in the Presence of Moving Obstacles8 Nov
- 2024Safety Filtering While Training: Improving the Performance and Sample Efficiency of Reinforcement Learning Agents15 Oct
Other references · 19
- 2017Safe Reinforcement Learning via Shieldingcited by 449
- 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 297
- 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 175
- 2017A General Safety Framework for Learning-Based Control in Uncertain Robotic Systemscited by 163
- 2018A predictive safety filter for learning-based control of constrained nonlinear dynamical systemscited by 118
- 2019Neural Simplex Architecturecited by 29
- 2022Sim-to-Lab-to-Real: Safe Reinforcement Learning with Shielding and Generalization Guaranteescited by 27
- 2022Safe Reinforcement Learning Using Black-Box Reachability Analysiscited by 26
- 2024SRL-VIC: A Variable Stiffness-Based Safe Reinforcement Learning for Contact-Rich Robotic Taskscited by 15
- 2023Interval Reachability of Nonlinear Dynamical Systems with Neural Network Controllerscited by 9
- 2015Feedback Refinement Relations for the Synthesis of Symbolic Controllerscited by 6
- 2017A Component-Based Simplex Architecture for High-Assurance Cyber-Physical Systemscited by 6
7 more
- 2023Safe Reinforcement Learning with Dead-Ends Avoidance and Recoverycited by 5
- 2023Dynamic Simplex: Balancing Safety and Performance in Autonomous Cyber Physical Systemscited by 4
- 2025Anti-Slip AI-Driven Model-Free Control with Global Exponential Stability in Skid-Steering Robotscited by 2
- 2025Synthesis of Deep Neural Networks with Safe Robust Adaptive Control for Reliable Operation of Wheeled Mobile Robotscited by 2
- 2025Reliable Robotic Task Execution in the Face of Anomaliescited by 2
- 2026NMPC-Augmented Visual Navigation and Safe Learning Control for Large-Scale Mobile Robotscited by 1
- 2026Vision-based Goal-Reaching Control for Mobile Robots Using a Hierarchical Learning Frameworkcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026REVERSAL-BENCH: A Reversibility Axis and Reset Oracle for Measuring the Reset-Free RL Cliff
- 2026RESETTLE: Robotic Recovery through Disagreement-Triggered Retrieval and Efficient Corrective Control ↗
- 2024RecoveryChaining: Learning Local Recovery Policies for Robust Manipulation
- 2026MAGMA-GEN: Validated Recovery Supervision from Ambiguous Failures via Counterfactual Re-Execution
- 2025Learning to Undo: Rollback-Augmented Reinforcement Learning with Reversibility Signals
- 2026Kintsugi-VLA: Turning Failed Robot Rollouts into Recovery Data through Interventional Recoverability
- 2026Is Online Interaction Necessary for Recovery? A Minimalist Approach to Robust Planning via Perturbation ↗
- 2026SafeExplorer: An Unbiased Policy Gradient for Reinforcement Learning with Recovery Interventions