# Instruction Adherence in Coding Agent Configuration Files: A Factorial Study of Four File-Structure Variables

arXiv: https://arxiv.org/abs/2605.10039 · HTML: https://mlsanity.com/paper/2605.10039/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 6 of its 6 references that are in the map.
- First posted on arXiv 11 May 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 6 of 6)

- [RULER: What's the Real Context Size of Your Long-Context Language Models?](https://mlsanity.com/paper/2404.06654/) (2024)
- [LIFBench: Evaluating the Instruction Following Performance and Stability of Large Language Models in Long-Context Scenarios](https://mlsanity.com/paper/2411.07037/) (2024)
- [Control Illusion: The Failure of Instruction Hierarchies in Large Language Models](https://mlsanity.com/paper/2502.15851/) (2025)
- [Context Is What You Need: The Maximum Effective Context Window for Real World Limits of LLMs](https://mlsanity.com/paper/2509.21361/) (2025)
- [Structured Context Engineering for File-Native Agentic Systems: Evaluating Schema Accuracy, Format Effectiveness, and Multi-File Navigation at Scale](https://mlsanity.com/paper/2602.05447/) (2026)
- [Long Context, Less Focus: A Scaling Gap in LLMs Revealed through Privacy and Personalization](https://mlsanity.com/paper/2602.15028/) (2026)

## Built on by (most cited first, 3 of 3)

- [Agentic Agile-V: From Vibe Coding to Verified Engineering in Software and Hardware Development](https://mlsanity.com/paper/2605.20456/) (2026)
- [Rule Taxonomy and Evolution in AI IDEs: A Mining and Survey Study](https://mlsanity.com/paper/2606.12231/) (2026)
- [Curating Always-Loaded Context for LLM Agents: A Capacitated Assortment Model with Censored Feedback](https://mlsanity.com/paper/2610.11007/) (2026)

## Abstract (arXiv)

Frontier coding agents read configuration files (CLAUDE.md, AGENTS.md, Cursor Rules) at session start and are expected to follow the conventions inside them. Practitioners assume that structural choices (file size, instruction position, file architecture, contradictions in adjacent files) measurably affect adherence. We report a systematic factorial study of these choices using four manipulated variables, measuring compliance with a trivial target annotation across 1,650 Claude Code CLI sessions (16,050 function-level observations) on two TypeScript codebases, three frontier models (primarily Sonnet 4.6, with Opus 4.6 as a CLI-matched cross-model check and Opus 4.7 reported descriptively under a CLI-version confound), and five coding tasks. We use mixed-effects models with a Bayesian companion. None of the four structural variables or three two-way interactions produces a detectable contrast after multiple-testing correction. Size and conflict nulls are supported by affirmative-null Bayes factors (BF10 between 0.05 and 0.10); position and architecture nulls are failures to reject without Bayes-factor support. The largest effect we measured is within-session: each additional function the agent generates is associated with approximately 5.6% lower odds of compliance per step (OR = 0.944) within the session-length range we tested, though the relationship is non-monotonic rather than a constant per-step effect. This reproduces on a second TypeScript codebase and on Opus 4.6 at matched configuration; it was identified during analysis rather than pre-specified. Within the conditions tested, file-structure variables did not produce detectable contrasts; compliance varies systematically between coding tasks and across each session's sequence of generated functions.