# Coding with "Enemy": Can Human Developers Detect AI Agent Sabotage?

arXiv: https://arxiv.org/abs/2606.05647 · HTML: https://mlsanity.com/paper/2606.05647/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 24 of its 36 references that are in the map.
- First posted on arXiv 4 Jun 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 24 of 24)

- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://mlsanity.com/paper/2404.07972/) (2024)
- [Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces](https://mlsanity.com/paper/2601.11868/) (2026)
- [AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents](https://mlsanity.com/paper/2410.09024/) (2024)
- [Alignment faking in large language models](https://mlsanity.com/paper/2412.14093/) (2024)
- [Frontier Models are Capable of In-context Scheming](https://mlsanity.com/paper/2412.04984/) (2024)
- [AI Control: Improving Safety Despite Intentional Subversion](https://mlsanity.com/paper/2312.06942/) (2023)
- [ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models](https://mlsanity.com/paper/2404.07738/) (2024)
- [Agentic Misalignment: How LLMs Could Be Insider Threats](https://mlsanity.com/paper/2510.05179/) (2025)
- [Natural Emergent Misalignment from Reward Hacking in Production RL](https://mlsanity.com/paper/2511.18397/) (2025)
- [CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale](https://mlsanity.com/paper/2506.02548/) (2025)
- [SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents](https://mlsanity.com/paper/2506.15740/) (2025)
- [Sabotage Evaluations for Frontier Models](https://mlsanity.com/paper/2410.21514/) (2024)
- [Agents of Chaos](https://mlsanity.com/paper/2602.20021/) (2026)
- [Reliable Weak-to-Strong Monitoring of LLM Agents](https://mlsanity.com/paper/2508.19461/) (2025)
- [Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agents](https://mlsanity.com/paper/2602.13379/) (2026)
- [Async Control: Stress-testing Asynchronous Control Measures for LLM Agents](https://mlsanity.com/paper/2512.13526/) (2025)
- [Professional Software Developers Don't Vibe, They Control: AI Agent Use for Coding in 2025](https://mlsanity.com/paper/2512.14012/) (2025)
- [LLMs Can Covertly Sandbag on Capability Evaluations Against Chain-of-Thought Monitoring](https://mlsanity.com/paper/2508.00943/) (2025)
- [Agentic Refactoring: An Empirical Study of AI Coding Agents](https://mlsanity.com/paper/2511.04824/) (2025)
- [BashArena: A Control Setting for Highly Privileged AI Agents](https://mlsanity.com/paper/2512.15688/) (2025)
- [CTRL-ALT-DECEIT: Sabotage Evaluations for Automated AI R&D](https://mlsanity.com/paper/2511.09904/) (2025)
- [How does information access affect LLM monitors' ability to detect sabotage?](https://mlsanity.com/paper/2601.21112/) (2026)
- [Optimizing AI Agent Attacks With Synthetic Data](https://mlsanity.com/paper/2511.02823/) (2025)

## Built on by (most cited first, 4 of 4)

- [ResearchArena: Evaluating Sabotage and Monitoring in Automated AI R&D](https://mlsanity.com/paper/2607.19321/) (2026)
- [AI Persuasion as a Threat to Human Control](https://mlsanity.com/paper/2609.14796/) (2026)
- [ParallelPilot: Supporting Coordination and Monitoring in Parallel AI Coding](https://mlsanity.com/paper/2609.33113/) (2026)
- [SecureVibe: Making Vibe Coding More Secure](https://mlsanity.com/paper/2609.38606/) (2026)

## Abstract (arXiv)

AI coding agents are increasingly embedded in real-world software development, collaborating with human developers while gaining broader access to codebases and tools. This creates a new attack surface: an agent can exploit human trust to sabotage development, for instance by inserting malicious code to accomplish a hidden side task. Most prior work studies AI sabotage in AI-only settings, paying limited attention to the role of human oversight in detecting and mitigating such malicious behavior. To address this gap, we conduct the first large-scale study of human oversight in AI coding sabotage. Over 100 participants collaborate with one of four frontier models (Claude-Opus-4.6, GPT-5.4, Gemini-3.1-Pro, and MiniMax-M2.7) on a long-horizon coding task lasting around five hours, designed to mimic real-world workflows. We find that 83/88 (94%) of developers in the no-monitor conditions fail to detect sabotage, and our analysis of participant feedback attributes this vulnerability to minimal code review, plausible cover story, and overtrust in agents. We further test the effectiveness of a safety monitor in one condition: while the monitor reduces sabotage success, sabotage still succeeds in 9/16 (56%) of sessions with a correct monitor alert. Drawing on participant feedback, we offer actionable suggestions for better monitor design. This work complements existing AI safety research and highlights an urgent need for human-centric safety mechanisms that account for human factors, particularly in long-horizon, real-world development settings.