# When Should a Human Take Back Control? Optimal Delegation under Turbulent AI Risk

arXiv: https://arxiv.org/abs/2609.32083 · HTML: https://mlsanity.com/paper/2609.32083/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 52 references that are in the map.
- First posted on arXiv 25 Sep 2026.

## Builds on (most cited first, 30 of 31)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Why Do Multi-Agent LLM Systems Fail?](https://mlsanity.com/paper/2503.13657/) (2025)
- [AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents](https://mlsanity.com/paper/2410.09024/) (2024)
- [Ragas: Automated Evaluation of Retrieval Augmented Generation](https://mlsanity.com/paper/2309.15217/) (2023)
- [How Language Model Hallucinations Can Snowball](https://mlsanity.com/paper/2305.13534/) (2023)
- [Consistent Estimators for Learning to Defer to an Expert](https://mlsanity.com/paper/2006.01862/) (2020)
- [Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer](https://mlsanity.com/paper/1711.06664/) (2017)
- [AI Control: Improving Safety Despite Intentional Subversion](https://mlsanity.com/paper/2312.06942/) (2023)
- [Is Self-Repair a Silver Bullet for Code Generation?](https://mlsanity.com/paper/2306.09896/) (2023)
- [OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety](https://mlsanity.com/paper/2507.06134/) (2025)
- [The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs](https://mlsanity.com/paper/2509.09677/) (2025)
- [Ctrl-Z: Controlling AI Agents via Resampling](https://mlsanity.com/paper/2504.10374/) (2025)
- [Scaling Laws For Scalable Oversight](https://mlsanity.com/paper/2504.18530/) (2025)
- [AgentHallu: Benchmarking Automated Hallucination Attribution of LLM-based Agents](https://mlsanity.com/paper/2601.06818/) (2026)
- [Hallucination Cascade: Analyzing Error Propagation in Multi-Agent LLM Systems](https://mlsanity.com/paper/2606.07937/) (2026)
- [Control randomisation approach for policy gradient and application to reinforcement learning in optimal switching](https://mlsanity.com/paper/2404.17939/) (2024)
- [Deterministic Policy Gradient for Reinforcement Learning with Continuous Time and State](https://mlsanity.com/paper/2509.23711/) (2025)
- [Learning to Defer in Congested Systems: The AI-Human Interplay](https://mlsanity.com/paper/2402.12237/) (2024)
- [Optimizing Delegation in Collaborative Human-AI Hybrid Teams](https://mlsanity.com/paper/2402.05605/) (2024)
- [Markov approximation for controlled Hawkes Jump-Diffusions with general kernels](https://mlsanity.com/paper/2507.11294/) (2025)
- [Randomisation with moral hazard: a path to existence of optimal contracts](https://mlsanity.com/paper/2311.13278/) (2023)
- [Optimal Impulse Control for Cyber Risk Management](https://mlsanity.com/paper/2410.17706/) (2024)
- [Agency Problems and Adversarial Bilevel Optimization under Uncertainty and Cyber Threats](https://mlsanity.com/paper/2505.08989/) (2025)
- [Open Problems in Frontier AI Risk Management](https://mlsanity.com/paper/2604.25982/) (2026)

## Abstract (arXiv)

Deploying AI systems requires deciding when to delegate tasks and when humans should intervene to monitor and mitigate risk induced by AI operations. These decisions are challenging when failures cluster: a hallucination or harmful output can trigger further errors, creating periods of elevated risk. We introduce a continuous-time framework for learning adaptive human oversight under such turbulent AI risk. Existing oversight and delegation formulations condition on history but do not model incident clustering, or its suppression by supervision effort, jointly with the delegation decision and this study fixes this gap. The self-exciting dynamics capture how risk events increase the likelihood of subsequent events, making their timing and history central to decision-making. We formulate a stochastic control problem combining human actions, monitoring effort, and switching between human-AI-assisted operation and full AI delegation, balancing operational rewards against oversight costs, and cascading AI-failures and induced uncertainty. Human participation is an endogenous component of risk management: the policy determines both when oversight is needed and how much effort to allocate. We study a relaxed switching formulation and propose Hawkes-PPO, a policy-gradient method that uses a bank of exponential filters of observed incident times. In a synthetic environment it attains a higher risk-adjusted objective than either fixed regime and approaches an approximate full-information oracle. We illustrate our results with numerical simulations by examining how cascade risks influence intervention and delegation, connecting reinforcement learning with adaptive human oversight of AI systems. In particular, we illustrate the benefit of our switching strategy and Hawkes-PPO algorithm to monitor the project efficiently along time, reducing turbulent risks occurrences and costs.