# Practical challenges of control monitoring in frontier AI deployments

arXiv: https://arxiv.org/abs/2512.22154 · HTML: https://mlsanity.com/paper/2512.22154/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 29 references that are in the map.
- First posted on arXiv 15 Dec 2025.

## Builds on (most cited first, 22 of 22)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering](https://mlsanity.com/paper/2410.07095/) (2024)
- [Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety](https://mlsanity.com/paper/2507.11473/) (2025)
- [AI Control: Improving Safety Despite Intentional Subversion](https://mlsanity.com/paper/2312.06942/) (2023)
- [Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming](https://mlsanity.com/paper/2501.18837/) (2025)
- [Large Language Model-Based Agents for Software Engineering: A Survey](https://mlsanity.com/paper/2409.02977/) (2024)
- [LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead](https://mlsanity.com/paper/2404.04834/) (2024)
- [Auditing language models for hidden objectives](https://mlsanity.com/paper/2503.10965/) (2025)
- [International AI Safety Report](https://mlsanity.com/paper/2501.17805/) (2025)
- [Safety Cases: How to Justify the Safety of Advanced AI Systems](https://mlsanity.com/paper/2403.10462/) (2024)
- [Ctrl-Z: Controlling AI Agents via Resampling](https://mlsanity.com/paper/2504.10374/) (2025)
- [A sketch of an AI control safety case](https://mlsanity.com/paper/2501.17315/) (2025)
- [Evaluating Frontier Models for Stealth and Situational Awareness](https://mlsanity.com/paper/2505.01420/) (2025)
- [Towards evaluations-based safety cases for AI scheming](https://mlsanity.com/paper/2411.03336/) (2024)
- [Games for AI Control: Models of Safety Evaluations of AI Deployment Protocols](https://mlsanity.com/paper/2409.07985/) (2024)
- [Adaptive Deployment of Untrusted LLMs Reduces Distributed Threats](https://mlsanity.com/paper/2411.17693/) (2024)
- [AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities](https://mlsanity.com/paper/2508.11126/) (2025)
- [How to evaluate control measures for LLM agents? A trajectory from today to superintelligence](https://mlsanity.com/paper/2504.05259/) (2025)
- [Safety case template for frontier AI: A cyber inability argument](https://mlsanity.com/paper/2411.08088/) (2024)
- [Subversion Strategy Eval: Can language models statelessly strategize to subvert control protocols?](https://mlsanity.com/paper/2412.12480/) (2024)
- [Combining Cost-Constrained Runtime Monitors for AI Safety](https://mlsanity.com/paper/2507.15886/) (2025)

## Built on by (most cited first, 8 of 8)

- [Training Agents to Self-Report Misbehavior](https://mlsanity.com/paper/2602.22303/) (2026)
- [Distributed Attacks in Persistent-State AI Control](https://mlsanity.com/paper/2607.02514/) (2026)
- [Multi-Agent AI Control: Distributed Attacks Hamper Per-Instance Monitors](https://mlsanity.com/paper/2607.07368/) (2026)
- [Classifier Context Rot: Monitor Performance Degrades with Context Length](https://mlsanity.com/paper/2605.12366/) (2026)
- [Retrying vs Resampling in AI Control](https://mlsanity.com/paper/2605.26047/) (2026)
- [Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages](https://mlsanity.com/paper/2607.06596/) (2026)
- [Red-Teaming Auto Mode: Improving Blocking Classifiers Against Malign Coding Agents](https://mlsanity.com/paper/2609.19587/) (2026)
- [Monitorability Disposition in Large Reasoning Models](https://mlsanity.com/paper/2610.04914/) (2026)

## Abstract (arXiv)

Automated control monitors could play an important role in overseeing highly capable AI agents that we do not fully trust. Prior work has explored control monitoring in simplified settings, but scaling monitoring to real-world deployments introduces additional dynamics: parallel agent instances, non-negligible oversight latency, incremental attacks between agent instances, and the difficulty of identifying scheming agents based on individual harmful actions. In this paper, we analyse design choices to address these challenges, focusing on three forms of monitoring with different latency-safety trade-offs: synchronous, semi-synchronous, and asynchronous monitoring. We introduce a high-level safety case sketch as a tool for understanding and comparing these monitoring protocols. Our analysis identifies three challenges -- oversight, latency, and recovery -- and explores them in four case studies of possible future AI deployments.