Classifier Context Rot: Monitor Performance Degrades with Context Length
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
Monitoring coding agents for dangerous behavior using language models requires classifying transcripts that often exceed 500K tokens, but prior agent monitoring benchmarks rarely contain transcripts longer than 100K tokens. We show that when used as classifiers, current frontier models fail to notice dangerous actions more often in longer transcripts. In particular, on a dataset that requires identifying when a coding agent takes a subtly dangerous action, Opus 4.6, GPT 5.4, and Gemini 3.1 miss these actions 2× to 30× more often when they occur after 800K tokens of benign activity than when they occur on their own. We also show that these weaknesses can be partially mitigated with prompting techniques such as periodic reminders throughout the transcript and may be mitigated further with better post-training. Monitor evaluations that do not consider long-context degradation are likely overestimating monitor performance.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 9 of its 15 references that are in the map. First posted on arXiv 12 May 2026.
Builds on
9 of 15 references.
Closest ancestors · 2
Same direction, newest first
- 2026Constitutional Black-Box Monitoring for Scheming in LLM Agents28 Feb
- 2025Adaptive Attacks on Trusted Monitors Subvert AI Control Protocols10 Oct
Other references · 7
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2023Effective Long-Context Scaling of Foundation Modelscited by 301
- 2023AI Control: Improving Safety Despite Intentional Subversioncited by 207
- 2025Measuring AI Ability to Complete Long Software Taskscited by 142
- 2024Can't Remember Details in Long Documents? You Need Some R&Rcited by 10
- 2025Practical challenges of control monitoring in frontier AI deploymentscited by 6
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026PrefixGuard: From LLM-Agent Traces to Online Failure-Warning Monitors
- 2026StateComp: Learning When to Compress History in Long Horizon Agents
- 2026Disentangling Task Difficulty from Run-Level Failure in Agent Failure Prediction
- 2026Strained Coherence: A Pre-Failure Signal in Coding Agent Execution Trajectories
- 2026Agent Safety From Within: Detecting Harmful Trajectories from LLM Internal States
- 2026PreAct-Bench: Benchmarking Predictive Monitoring in LLMs
- 2026Trained Agentic Context Management
- 2026Prompted to Discriminate: Generalizing Malicious-Input Probes in the Wild