# Reversible Simplex Supervision with Post-Action Debt Accounting for Goal-Reaching RL

arXiv: https://arxiv.org/abs/2601.19499 · HTML: https://mlsanity.com/paper/2601.19499/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 21 of its 30 references that are in the map.
- First posted on arXiv 27 Jan 2026.

## Placed in

- Deep reinforcement learning methods / [Safe and constrained RL](https://mlsanity.com/deeprl/safe-and-constrained-rl/)

## Builds on (most cited first, 21 of 21)

- [Safe Reinforcement Learning via Shielding](https://mlsanity.com/paper/1708.08611/) (2017)
- [A Lyapunov-based Approach to Safe Reinforcement Learning](https://mlsanity.com/paper/1805.07708/) (2018)
- [Recovery RL: Safe Reinforcement Learning with Learned Recovery Zones](https://mlsanity.com/paper/2010.15920/) (2020)
- [A General Safety Framework for Learning-Based Control in Uncertain Robotic Systems](https://mlsanity.com/paper/1705.01292/) (2017)
- [A predictive safety filter for learning-based control of constrained nonlinear dynamical systems](https://mlsanity.com/paper/1812.05506/) (2018)
- [Neural Simplex Architecture](https://mlsanity.com/paper/1908.00528/) (2019)
- [Sim-to-Lab-to-Real: Safe Reinforcement Learning with Shielding and Generalization Guarantees](https://mlsanity.com/paper/2201.08355/) (2022)
- [Safe Reinforcement Learning Using Black-Box Reachability Analysis](https://mlsanity.com/paper/2204.07417/) (2022)
- [Safety Filtering While Training: Improving the Performance and Sample Efficiency of Reinforcement Learning Agents](https://mlsanity.com/paper/2410.11671/) (2024)
- [SRL-VIC: A Variable Stiffness-Based Safe Reinforcement Learning for Contact-Rich Robotic Tasks](https://mlsanity.com/paper/2406.13744/) (2024)
- [Interval Reachability of Nonlinear Dynamical Systems with Neural Network Controllers](https://mlsanity.com/paper/2301.07912/) (2023)
- [Feedback Refinement Relations for the Synthesis of Symbolic Controllers](https://mlsanity.com/paper/1503.03715/) (2015)
- [A Component-Based Simplex Architecture for High-Assurance Cyber-Physical Systems](https://mlsanity.com/paper/1704.04759/) (2017)
- [Safe Reinforcement Learning with Dead-Ends Avoidance and Recovery](https://mlsanity.com/paper/2306.13944/) (2023)
- [Dynamic Simplex: Balancing Safety and Performance in Autonomous Cyber Physical Systems](https://mlsanity.com/paper/2302.09750/) (2023)
- [Anti-Slip AI-Driven Model-Free Control with Global Exponential Stability in Skid-Steering Robots](https://mlsanity.com/paper/2504.08831/) (2025)
- [Synthesis of Deep Neural Networks with Safe Robust Adaptive Control for Reliable Operation of Wheeled Mobile Robots](https://mlsanity.com/paper/2508.10634/) (2025)
- [Reliable Robotic Task Execution in the Face of Anomalies](https://mlsanity.com/paper/2510.23121/) (2025)
- [Safe Reinforcement Learning of Robot Trajectories in the Presence of Moving Obstacles](https://mlsanity.com/paper/2411.05784/) (2024)
- [NMPC-Augmented Visual Navigation and Safe Learning Control for Large-Scale Mobile Robots](https://mlsanity.com/paper/2601.00609/) (2026)
- [Vision-based Goal-Reaching Control for Mobile Robots Using a Hierarchical Learning Framework](https://mlsanity.com/paper/2601.00610/) (2026)

## Abstract (arXiv)

Deploying reinforcement learning (RL) on multi-tonne robots calls for supervisory mechanisms that address both operational safety and progress toward task completion. However, repeated switching need not preserve task progress when a learned action increases storage before recovery takes control. We introduce reversible Simplex supervision with post-action debt accounting for a frozen finite-state policy and robust-adaptive recovery. Under exact sampled-state information and stated model and certificate conditions, we prove that recovery repayment exceeding a uniform triggering-edge debt bound guarantees finite switching and finite-sample goal entry. We formulate reachability-based certificate constructions for establishing these sufficient conditions. In 20 matched simulations, goal-entry counts are 20 with debt gating and 18 without it; the two remaining runs terminate under the supervisor's admissibility stopping rule. On an experimental 6000 kg robot, 24 asphalt and soft-terrain trials evaluate 50 ms supervision above a 1 kHz actuator stack; all eight triggered recoveries complete debt-gated re-entry. The experiments demonstrate the supervisory mechanism in the tested trials.