mlsanityData to 14 Oct 2026

When and How Context Rot Appears in Coding Agents: A White-Box Study of Agent Skills in Code Auditing

arXiv 2607.17937 ↗First posted 20 Jul 2026cs.SEcited by 0

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Agent Skills package procedural instructions and checks for use by general-purpose agents, but loading a skill does not guarantee that every requirement remains active throughout a long tool-using trajectory. We study this problem in a production-derived, white-box code-audit workflow. Holding the task and 24 artifact checks fixed, we vary the surrounding context and classify where failures first become visible: lost requirements, editing drift, failed checking, or non-agent evaluator/runtime failures. Codex with gpt-5.4-mini passes 8/10 runs in a 10,991-character clean context but only 3/10 in both a 299,140-character relevant context and an equal-length irrelevant context. This 50-percentage-point difference is large but remains trend-level under two-sided Fisher tests (p = 0.0698). Requirement coverage nevertheless stays above 92% in both long conditions, showing that a few omissions can invalidate an otherwise complete artifact. A second task passes all clean and long runs, so the evidence does not support a universal context-length threshold. A detailed external checklist passes 10/10 runs, compared with 5/10 for a generic self-check (p = 0.0325). Coding-agent scaffolds may help by selecting a smaller working set, but they do not eliminate failures. We do not introduce context rot or a new general monitoring method; we provide a bounded failure classification and empirical case study for white-box code auditing.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 11 of its 33 references that are in the map. First posted on arXiv 20 Jul 2026.

Builds on

11 of 33 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026Failure as a Process: An Anatomy of CLI Coding Agent Trajectories10 Jul
  2. 2026How Coding Agents Fail Their Users: A Large-Scale Analysis of Developer-Agent Misalignment in 20,574 Real-World Sessions28 May
  3. 2025Understanding Code Agent Behaviour: An Empirical Study of Success and Failure Trajectories31 Oct
  4. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agents23 Jul

Other references · 7

  1. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  2. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  3. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  4. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
  5. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  6. 2026Coding Agents are Effective Long-Context Processorscited by 12
  7. 2026Context Rot in AI-Assisted Software Development: Repurposing Documentation Consistency for AI Configuration Artifactscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Agent READMEs: An Empirical Study of Context Files for Agentic Coding
  2. 2026Requirements After the First Edit: Mining Late Requirement Emergence and Rework in Real-World Coding-Agent Sessions
  3. 2026What Coding Agents Leave Unsaid: Measuring Task-State Gaps Between Execution and Reports
  4. 2026Guardrails Beat Guidance: A Large-Scale Study of Rules, Skills, and Persistent Configuration for Coding Agents
  5. 2026Code Understanding is a Bottleneck for Coding Agents
  6. 2026From Agent Behaviour to Agent-Friendly Documentation: An Empirical Study of How Coding Agents Discover, Read, and Write Technical Documentation
  7. 2026ContextBench: A Benchmark for Context Retrieval in Coding Agents
  8. 2026Beyond Resolution Rates: Behavioral Drivers of Coding Agent Success and Failure