# SLEIGHT-Bench: A Benchmark of Evasion Attacks Against Agent Monitors

arXiv: https://arxiv.org/abs/2605.16626 · HTML: https://mlsanity.com/paper/2605.16626/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 10 of its 21 references that are in the map.
- First posted on arXiv 15 May 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 10 of 10)

- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [AI Control: Improving Safety Despite Intentional Subversion](https://mlsanity.com/paper/2312.06942/) (2023)
- [Measuring AI Ability to Complete Long Software Tasks](https://mlsanity.com/paper/2503.14499/) (2025)
- [Is Power-Seeking AI an Existential Risk?](https://mlsanity.com/paper/2206.13353/) (2022)
- [Reliable Weak-to-Strong Monitoring of LLM Agents](https://mlsanity.com/paper/2508.19461/) (2025)
- [Async Control: Stress-testing Asynchronous Control Measures for LLM Agents](https://mlsanity.com/paper/2512.13526/) (2025)
- [BashArena: A Control Setting for Highly Privileged AI Agents](https://mlsanity.com/paper/2512.15688/) (2025)
- [On the Impossibility of Separating Intelligence from Judgment: The Computational Intractability of Filtering for AI Alignment](https://mlsanity.com/paper/2507.07341/) (2025)
- [Bypassing Prompt Guards in Production with Controlled-Release Prompting](https://mlsanity.com/paper/2510.01529/) (2025)
- [MonitoringBench: Semi-Automated Red-Teaming for Agent Monitoring](https://mlsanity.com/paper/2605.09684/) (2026)

## Built on by (most cited first, 6 of 6)

- [Reward Hacking Challenges Oversight of Autonomous Research Agents](https://mlsanity.com/paper/2609.28614/) (2026)
- [RH-Detect: A Unified Benchmark for Reward Hacking Detection](https://mlsanity.com/paper/2610.10947/) (2026)
- [Red-Teaming Auto Mode: Improving Blocking Classifiers Against Malign Coding Agents](https://mlsanity.com/paper/2609.19587/) (2026)
- [When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systems](https://mlsanity.com/paper/2609.23700/) (2026)
- [From Probe Scores to Alarm Policies: Operational Validity of Activation Monitors for Language-Model Agents](https://mlsanity.com/paper/2610.04575/) (2026)
- [Towards a Unified Misuse Monitoring Benchmark](https://mlsanity.com/paper/2610.07089/) (2026)

## Abstract (arXiv)

Since autonomous coding agents generate complex behaviors at high-volume, we may want to use other LLMs to monitor actions to reduce the risk from dangerous misaligned behavior. To better understand the limitations of such monitors against the diverse attack strategies that a coding agent could use, we present SLEIGHT-Bench (Subtle Low-itEration Insight-Guided Harmful Transcripts), a benchmark of synthetic transcripts containing 40 attacks across 11 categories, each showing a coding agent covertly pursuing a harmful objective (e.g. weight exfiltration, credential theft, unmonitored rogue deployment). We find that 20 of 40 attacks are never caught by an Opus 4.6 monitor with extended thinking across 10 trials at a 1% false-positive rate, while the overall catch rate is 32%. We identify several evasion strategies such as exploiting model priors, user instruction ambiguity, and state manipulation, which facilitate creating attacks that evade frontier monitors. We also elicit stronger monitor performance using coding agents as monitors versus regular prompted monitors, and for some evasion strategies show improved catch rates with targeted monitor prompts. Our dataset and evaluation framework are available at https://github.com/safety-research/sleight-bench and https://huggingface.co/datasets/sleightbench/SLEIGHT-Bench.