# Agent READMEs: An Empirical Study of Context Files for Agentic Coding

arXiv: https://arxiv.org/abs/2511.12884 · HTML: https://mlsanity.com/paper/2511.12884/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 47 of its 86 references that are in the map.
- First posted on arXiv 17 Nov 2025.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 30 of 47)

- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [A Systematic Evaluation of Large Language Models of Code](https://mlsanity.com/paper/2202.13169/) (2022)
- [RepairAgent: An Autonomous, LLM-Based Agent for Program Repair](https://mlsanity.com/paper/2403.17134/) (2024)
- [Repository-Level Prompt Generation for Large Language Models of Code](https://mlsanity.com/paper/2206.12839/) (2022)
- [SpecRover: Code Intent Extraction via LLMs](https://mlsanity.com/paper/2408.02232/) (2024)
- [Model Context Protocol (MCP) at First Glance: Studying the Security and Maintainability of MCP Servers](https://mlsanity.com/paper/2506.13538/) (2025)
- [On the Robustness of Code Generation Techniques: An Empirical Study on GitHub Copilot](https://mlsanity.com/paper/2302.00438/) (2023)
- [Agents in Software Engineering: Survey, Landscape, and Vision](https://mlsanity.com/paper/2409.09030/) (2024)
- [Can Large Language Models Transform Natural Language Intent into Formal Method Postconditions?](https://mlsanity.com/paper/2310.01831/) (2023)
- [Can LLMs Replace Manual Annotation of Software Engineering Artifacts?](https://mlsanity.com/paper/2408.05534/) (2024)
- [Human-In-the-Loop Software Development Agents](https://mlsanity.com/paper/2411.12924/) (2024)
- [Agentic Software Engineering: Foundational Pillars and a Research Roadmap](https://mlsanity.com/paper/2509.06216/) (2025)
- [An Empirical Study on Usage and Perceptions of LLMs in a Software Engineering Project](https://mlsanity.com/paper/2401.16186/) (2024)
- [Search-Based LLMs for Code Optimization](https://mlsanity.com/paper/2408.12159/) (2024)
- [AutoDev: Automated AI-Driven Development](https://mlsanity.com/paper/2403.08299/) (2024)
- [HumanEvalComm: Benchmarking the Communication Competence of Code Generation for LLMs and LLM Agent](https://mlsanity.com/paper/2406.00215/) (2024)
- [On the Impacts of Contexts on Repository-Level Code Generation](https://mlsanity.com/paper/2406.11927/) (2024)
- [Evaluating Code Summarization Techniques: A New Metric and an Empirical Characterization](https://mlsanity.com/paper/2312.15475/) (2023)
- [A Pair Programming Framework for Code Generation via Multi-Plan Exploration and Feedback-Driven Refinement](https://mlsanity.com/paper/2409.05001/) (2024)
- [Categorizing the Content of GitHub README Files](https://mlsanity.com/paper/1802.06997/) (2018)
- [Agentic Refactoring: An Empirical Study of AI Coding Agents](https://mlsanity.com/paper/2511.04824/) (2025)
- [ITER: Iterative Neural Repair for Multi-Location Patches](https://mlsanity.com/paper/2304.12015/) (2023)
- [From Misuse to Mastery: Enhancing Code Generation with Knowledge-Driven AI Chaining](https://mlsanity.com/paper/2309.15606/) (2023)
- [Are LLMs Correctly Integrated into Software Systems?](https://mlsanity.com/paper/2407.05138/) (2024)
- [On the Use of Agentic Coding Manifests: An Empirical Study of Claude Code](https://mlsanity.com/paper/2509.14744/) (2025)
- [How to Refactor this Code? An Exploratory Study on Developer-ChatGPT Refactoring Conversations](https://mlsanity.com/paper/2402.06013/) (2024)
- [Can ChatGPT Support Developers? An Empirical Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2402.11702/) (2024)

## Built on by (most cited first, 23 of 23)

- [Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?](https://mlsanity.com/paper/2602.11988/) (2026)
- [Harness Engineering for Agentic AI Coding Tools: An Exploratory Study](https://mlsanity.com/paper/2602.14690/) (2026)
- [ContextCov: Deriving and Enforcing Executable Constraints from Agent Instruction Files](https://mlsanity.com/paper/2603.00822/) (2026)
- [From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI](https://mlsanity.com/paper/2606.14502/) (2026)
- [Codified Context: Infrastructure for AI Agents in a Complex Codebase](https://mlsanity.com/paper/2602.20478/) (2026)
- [Getting Better at Working With You: Compiling User Corrections into Runtime Enforcement for Coding Agents](https://mlsanity.com/paper/2606.13174/) (2026)
- [Knowledge Activation: AI Skills as the Institutional Knowledge Primitive for Agentic Software Development](https://mlsanity.com/paper/2603.14805/) (2026)
- [A Dataset of Agentic AI Coding Tool Configurations](https://mlsanity.com/paper/2605.08435/) (2026)
- [FAVA: Formal Authorization for Verified Agents with Evidence-Backed Permission Graphs](https://mlsanity.com/paper/2607.27267/) (2026)
- [Why Does CLAUDE.md Keep Growing? Catastrophic Remembering in Agentic Coding](https://mlsanity.com/paper/2608.11095/) (2026)
- [The Devil Is in the Interface: Evaluating How Tool Architecture Shapes Coding Agent Behavior](https://mlsanity.com/paper/2608.11386/) (2026)
- [Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Model](https://mlsanity.com/paper/2608.13867/) (2026)
- [From Registry to Repository: How AI Agent Skills Are Written, Adapted, and Maintained](https://mlsanity.com/paper/2607.00911/) (2026)
- [Rule Taxonomy and Evolution in AI IDEs: A Mining and Survey Study](https://mlsanity.com/paper/2606.12231/) (2026)
- [Probe-and-Refine Tuning of Repository Guidance for Coding Agents](https://mlsanity.com/paper/2606.20512/) (2026)
- [An Exploratory Study of Agent Plans for Agentic AI Coding Tools in Open-Source Software](https://mlsanity.com/paper/2608.04661/) (2026)
- [BootstrapAgent: Distilling Repository Setup into Reusable Agent Knowledge](https://mlsanity.com/paper/2605.15815/) (2026)
- [Toward Instructions-as-Code: Understanding the Impact of Instruction Files on Agentic Pull Requests](https://mlsanity.com/paper/2606.13449/) (2026)
- [From Agent Behaviour to Agent-Friendly Documentation: An Empirical Study of How Coding Agents Discover, Read, and Write Technical Documentation](https://mlsanity.com/paper/2608.20195/) (2026)
- [Who Maintains Agent Skills? A Longitudinal Study of Human-Governed, AI-Assisted Skill Maintenance](https://mlsanity.com/paper/2609.05677/) (2026)
- [Scanning the Harness: A Repository Study of Configuration Exposures in AI Coding Agents](https://mlsanity.com/paper/2609.07360/) (2026)
- [Specifying and Maintaining Agentic Workflows: An Empirical Study of GitHub Agentic Workflows](https://mlsanity.com/paper/2609.27263/) (2026)
- [Scientific Agents: Evaluating Profession-Specific System Prompts on Scientific Tasks](https://mlsanity.com/paper/2610.00084/) (2026)

## Abstract (arXiv)

Agentic coding tools receive goals written in natural language, break them down into specific tasks, and write or execute code with minimal human intervention. Central to this process are agent context files (e.g., AGENTS.md and CLAUDE.md) that provide persistent, project-level instructions. In this paper, we conduct the first large-scale empirical study of 2,303 agent context files from 1,925 repositories to characterize their structure, maintenance, and content. We find that these files are not static documentation but complex, difficultto-read artifacts that evolve like configuration code through frequent, small additions. Our content analysis of 16 instruction types shows that developers prioritize functional context, such as test procedures (75.9%), implementation details (70.8%), and architecture (68.1%). We also identify a significant gap: non-functional requirements such as security (14.8%) and performance (14.5%) are rarely specified. These findings indicate that while developers use context files to make agents functional, they provide few guardrails to ensure that agent-written code is secure or performant, highlighting the need for improved tools and practices.