Distributed Attacks in Persistent-State AI Control
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
As AI coding agents become more autonomous, they increasingly ship code iteratively, with the codebase persisting across sessions. This persistence creates a new attack surface: a misaligned or prompt-injected agent can distribute attacks across pull requests (PRs) and time its payload for the PR with the best natural cover. To study the resulting dynamics, we introduce Iterative VibeCoding, a setting for AI control, the study of safely deploying capable but potentially untrusted AI. In Iterative VibeCoding, a coding agent builds software over a sequence of PRs in a persistent codebase while pursuing a covert side task. Our benchmark includes two task families: CLI tools and Flask web services, across 20 total task variations. We use Claude Sonnet 4.5 as the attack agent and GPT-4o as the monitor. We compare gradual attacks, which distribute the side task across PRs, against non-gradual attacks concentrated in a single PR. No single monitor is robust to both: which strategy evades best (success while evading the monitor) depends on the monitor type, so a defender cannot close off both gradual and non-gradual attacks with any one monitor. High evasion (>= 65%) generalizes across model attack agent backends (Sonnet 4.5, Gemini 3.1 Pro, Kimi K2.5), confirming this is a property of the persistent-state attack surface rather than a single model's capability. Evasion also remains high across state-of-the-art monitor models and the gap between gradual and non-gradual evasion widens for more capable models. We introduce a stateful link-tracker monitor that tracks suspicious buildup across PRs. On both task families, it detects gradual attacks substantially better than diff monitors that merely see more accumulated history. Combining this stronger monitor with trajectory monitors in a four-monitor ensemble reduces gradual-attack evasion from 93% under the weakest standard diff monitor to 47%.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 24 of its 32 references that are in the map. First posted on arXiv 2 Jul 2026.
Builds on
24 of 32 references.
Closest ancestors · 6
Same direction, newest first
- 2026LinuxArena: A Control Setting for AI Agents in Live Production Software Environments16 Apr
- 2026How does information access affect LLM monitors' ability to detect sabotage?28 Jan
- 2025Async Control: Stress-testing Asynchronous Control Measures for LLM Agents15 Dec
- 2025SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents17 Jun
- 2025How to evaluate control measures for LLM agents? A trajectory from today to superintelligence7 Apr
- 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training10 Jan
Other references · 18
- 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 338
- 2024Frontier Models are Capable of In-context Schemingcited by 252
- 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetycited by 222
- 2023AI Control: Improving Safety Despite Intentional Subversioncited by 207
- 2023Large Language Models can Strategically Deceive their Users when Put Under Pressurecited by 132
- 2023Scheming AIs: Will AIs fake alignment during training in order to get power?cited by 90
- 2022Taxonomy of Attacks on Open-Source Software Supply Chainscited by 57
- 2025CoT Red-Handed: Stress Testing Chain-of-Thought Monitoringcited by 49
- 2024Sabotage Evaluations for Frontier Modelscited by 45
- 2025Ctrl-Z: Controlling AI Agents via Resamplingcited by 40
- 2024Games for AI Control: Models of Safety Evaluations of AI Deployment Protocolscited by 28
- 2024Adaptive Deployment of Untrusted LLMs Reduces Distributed Threatscited by 27
6 more
- 2024Subversion Strategy Eval: Can language models statelessly strategize to subvert control protocols?cited by 14
- 2025Strategic Dishonesty Can Undermine AI Safety Evaluations of Frontier LLMscited by 13
- 2025Practical challenges of control monitoring in frontier AI deploymentscited by 6
- 2026Attack Selection Reduces Safety in Concentrated AI Control Settings against Trusted Monitoringcited by 6
- 2025Wolves in the Repository: A Software Engineering Analysis of the XZ Utils Supply Chain Attackcited by 5
- 2024On the critical path to implant backdoors and the effectiveness of potential mitigation techniques: Early learnings from XZcited by 3
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Security in the Age of AI Teammates: An Empirical Study of Agentic Pull Requests on GitHub
- 2026Prompt Injection Attacks on Agentic Coding Assistants: A Systematic Analysis of Vulnerabilities in Skills, Tools, and Protocol Ecosystems
- 2026A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Framework
- 2026SynChain: Inducing Computer-Use Agent Systems to Construct Their Own Attack Chains
- 2026The Balkanization of Execution-Security Research for AI Coding Agents: Isolation, Access Control, and Time-of-Check-to-Time-of-Use Vulnerabilities
- 2026SoK: AI Secure Code Generation: Progress, Pitfalls, and Paths Forward
- 2026AgentWard: A Lifecycle Security Architecture for Autonomous AI Agents
- 2026The Best-Laid SCHEMEs: Coordinated Sabotage and Monitoring in Multi-Agent Systems