# Cutting Your Losses: Learning Fault-Tolerant Control and Optimal Stopping under Adverse Risk

arXiv: https://arxiv.org/abs/1902.05045 · HTML: https://mlsanity.com/paper/1902.05045/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Feb 2019.

## Built on by (most cited first, 5 of 5)

- [Learning to Shape Rewards using a Game of Two Partners](https://mlsanity.com/paper/2103.09159/) (2021)
- [DESTA: A Framework for Safe Reinforcement Learning with Markov Games of Intervention](https://mlsanity.com/paper/2110.14468/) (2021)
- [Timing is Everything: Learning to Act Selectively with Costly Actions and Budgetary Constraints](https://mlsanity.com/paper/2205.15953/) (2022)
- [All Language Models Large and Small](https://mlsanity.com/paper/2402.12061/) (2024)
- [SEREN: Knowing When to Explore and When to Exploit](https://mlsanity.com/paper/2205.15064/) (2022)

## Abstract (arXiv)

Recently, there has been a surge in interest in safe and robust techniques within reinforcement learning (RL). Current notions of risk in RL fail to capture the potential for systemic failures such as abrupt stoppages from system failures or surpassing of safety thresholds and the appropriate responsive controls in such instances. We propose a novel approach to risk minimisation within RL in which, in addition to taking actions that maximise its expected return, the controller learns a policy that is robust against stoppages due to an adverse event such as an abrupt failure. The results of the paper cover fault-tolerant control in textitworst-case scenarios under random stopping and optimal stopping, all in unknown environments. By demonstrating that the class of problems is represented by a variant of stochastic games, we prove the existence of a solution which is a unique fixed point equilibrium of the game and characterise the optimal controller behaviour. We then introduce a value function approximation algorithm that converges to the solution through simulation in unknown environments.