# Ctrl-Z: Controlling AI Agents via Resampling

arXiv: https://arxiv.org/abs/2504.10374 · HTML: https://mlsanity.com/paper/2504.10374/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 36 references that are in the map.
- First posted on arXiv 14 Apr 2025.

## Builds on (most cited first, 21 of 21)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [The Limitations of Deep Learning in Adversarial Settings](https://mlsanity.com/paper/1511.07528/) (2015)
- [Evasion Attacks against Machine Learning at Test Time](https://mlsanity.com/paper/1708.06131/) (2017)
- [Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision](https://mlsanity.com/paper/2312.09390/) (2023)
- [AI safety via debate](https://mlsanity.com/paper/1805.00899/) (2018)
- [Alignment faking in large language models](https://mlsanity.com/paper/2412.14093/) (2024)
- [Poisoning Web-Scale Training Datasets is Practical](https://mlsanity.com/paper/2302.10149/) (2023)
- [AI Control: Improving Safety Despite Intentional Subversion](https://mlsanity.com/paper/2312.06942/) (2023)
- [Measuring Progress on Scalable Oversight for Large Language Models](https://mlsanity.com/paper/2211.03540/) (2022)
- [RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts](https://mlsanity.com/paper/2411.15114/) (2024)
- [Harms from Increasingly Agentic Algorithmic Systems](https://mlsanity.com/paper/2302.10329/) (2023)
- [Scheming AIs: Will AIs fake alignment during training in order to get power?](https://mlsanity.com/paper/2311.08379/) (2023)
- [Adversarial Robustness of Deep Neural Networks: A Survey from a Formal Verification Perspective](https://mlsanity.com/paper/2206.12227/) (2022)
- [Games for AI Control: Models of Safety Evaluations of AI Deployment Protocols](https://mlsanity.com/paper/2409.07985/) (2024)
- [Adaptive Deployment of Untrusted LLMs Reduces Distributed Threats](https://mlsanity.com/paper/2411.17693/) (2024)
- [A Research Agenda: Dynamic Models to Defend Against Correlated Attacks](https://mlsanity.com/paper/1903.06293/) (2019)

## Built on by (most cited first, 30 of 44)

- [SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents](https://mlsanity.com/paper/2506.15740/) (2025)
- [Reliable Weak-to-Strong Monitoring of LLM Agents](https://mlsanity.com/paper/2508.19461/) (2025)
- [Evaluating Control Protocols for Untrusted AI Agents](https://mlsanity.com/paper/2511.02997/) (2025)
- [Control Tax: The Price of Keeping AI in Check](https://mlsanity.com/paper/2506.05296/) (2025)
- [Reasoning Models Sometimes Output Illegible Chains of Thought](https://mlsanity.com/paper/2510.27338/) (2025)
- [Agentic Uncertainty Reveals Agentic Overconfidence](https://mlsanity.com/paper/2602.06948/) (2026)
- [Practical challenges of control monitoring in frontier AI deployments](https://mlsanity.com/paper/2512.22154/) (2025)
- [LinuxArena: A Control Setting for AI Agents in Live Production Software Environments](https://mlsanity.com/paper/2604.15384/) (2026)
- [Optimizing AI Agent Attacks With Synthetic Data](https://mlsanity.com/paper/2511.02823/) (2025)
- [Training Agents to Self-Report Misbehavior](https://mlsanity.com/paper/2602.22303/) (2026)
- [Ensemble Monitoring for AI Control: Diverse Signals Outweigh More Compute](https://mlsanity.com/paper/2605.15377/) (2026)
- [Chain-of-thought obfuscation learned from output supervision can generalise to unseen tasks](https://mlsanity.com/paper/2601.23086/) (2026)
- [Distributed Attacks in Persistent-State AI Control](https://mlsanity.com/paper/2607.02514/) (2026)
- [MonitoringBench: Semi-Automated Red-Teaming for Agent Monitoring](https://mlsanity.com/paper/2605.09684/) (2026)
- [Training Deliberative Monitors for Black-Box Scheming Detection](https://mlsanity.com/paper/2605.29601/) (2026)
- [Calibrated Predictive Lower Bounds on Time-to-Unsafe-Sampling in LLMs](https://mlsanity.com/paper/2506.13593/) (2025)
- [Subversion via Focal Points: Investigating Collusion in LLM Monitoring](https://mlsanity.com/paper/2507.03010/) (2025)
- [Localized Calibrated Uncertainty in Code Language Models](https://mlsanity.com/paper/2512.24560/) (2025)
- [Basic Legibility Protocols Improve Trusted Monitoring](https://mlsanity.com/paper/2602.10153/) (2026)
- [CIAware-Bench: Benchmarking Control Intervention Awareness Across Frontier LLMs](https://mlsanity.com/paper/2606.11063/) (2026)
- [Against racing to AGI: Cooperation, deterrence, and catastrophic risks](https://mlsanity.com/paper/2507.21839/) (2025)
- [ResearchArena: Evaluating Sabotage and Monitoring in Automated AI R&D](https://mlsanity.com/paper/2607.19321/) (2026)
- [trajectory-judge: What Outcome-Only LLM Judges Miss on Agent Trajectories](https://mlsanity.com/paper/2609.00038/) (2026)
- [Retrying vs Resampling in AI Control](https://mlsanity.com/paper/2605.26047/) (2026)
- [Attack Selection in Agentic AI Control Evaluations Meaningfully Decreases Safety](https://mlsanity.com/paper/2606.06529/) (2026)
- [Bootstrapped Monitoring: Leveraging Transparent Reasoning to Oversee Stronger AI Agents](https://mlsanity.com/paper/2606.11998/) (2026)
- [Model Forensics: Investigating Whether Concerning Behavior Reflects Misalignment](https://mlsanity.com/paper/2606.26071/) (2026)
- [More Rejective, Not More Discriminative: The Unit of Verification in Pre-Execution LLM Oversight](https://mlsanity.com/paper/2608.23941/) (2026)
- [Reliability Theory for AI Control](https://mlsanity.com/paper/2609.26419/) (2026)
- [Covert Assistance: Helpful LLM Agents Evade Oversight in Multi-Agent Systems](https://mlsanity.com/paper/2609.39050/) (2026)

## Abstract (arXiv)

Control evaluations measure whether monitoring and security protocols for AI systems prevent intentionally subversive AI models from causing harm. Our work presents the first control evaluation performed in an agent environment. We construct BashBench, a dataset of 257 challenging multi-step system administration tasks, and evaluate whether various safety measures can prevent an adversarially constructed AI agent from covertly downloading and executing malicious code in this environment. This multi-step setting introduces new attack and defense dynamics, which we investigate in order to design novel control protocols that prevent safety failures without hindering the ability of non-malicious agents to perform useful work. We introduce a class of control protocols called resample protocols that dynamically take additional samples of certain actions. We find these protocols significantly improve on existing techniques by selectively blocking the AI agent from executing suspicious code and incriminating the agent by generating additional examples of dangerous behavior. We measure the tradeoff between attack prevention and usefulness; our best protocol combines resampling with analysis of previous steps, reducing the success rate of attacks from 58% to 7% at a 5% cost to the performance of a non-malicious agent.