Agent READMEs: An Empirical Study of Context Files for Agentic Coding
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Agentic coding tools receive goals written in natural language, break them down into specific tasks, and write or execute code with minimal human intervention. Central to this process are agent context files (e.g., AGENTS.md and CLAUDE.md) that provide persistent, project-level instructions. In this paper, we conduct the first large-scale empirical study of 2,303 agent context files from 1,925 repositories to characterize their structure, maintenance, and content. We find that these files are not static documentation but complex, difficultto-read artifacts that evolve like configuration code through frequent, small additions. Our content analysis of 16 instruction types shows that developers prioritize functional context, such as test procedures (75.9%), implementation details (70.8%), and architecture (68.1%). We also identify a significant gap: non-functional requirements such as security (14.8%) and performance (14.5%) are rarely specified. These findings indicate that while developers use context files to make agents functional, they provide few guardrails to ensure that agent-written code is secure or performant, highlighting the need for improved tools and practices.
Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 47 of its 86 references that are in the map. First posted on arXiv 17 Nov 2025.
Builds on
47 of 86 references.
Closest ancestors · 8
Same direction, newest first
- 2025Agentic Refactoring: An Empirical Study of AI Coding Agents6 Nov
- 2025On the Use of Agentic Coding Manifests: An Empirical Study of Claude Code18 Sep
- 2025Agentic Software Engineering: Foundational Pillars and a Research Roadmap7 Sep
- 2024Human-In-the-Loop Software Development Agents19 Nov
- 2024Agents in Software Engineering: Survey, Landscape, and Vision13 Sep
- 2024SpecRover: Code Intent Extraction via LLMs5 Aug
- 2024HumanEvalComm: Benchmarking the Communication Competence of Code Generation for LLMs and LLM Agent31 May
- 2024RepairAgent: An Autonomous, LLM-Based Agent for Program Repair25 Mar
Other references · 39
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
- 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
- 2022A Systematic Evaluation of Large Language Models of Codecited by 363
- 2022Repository-Level Prompt Generation for Large Language Models of Codecited by 106
- 2025Model Context Protocol (MCP) at First Glance: Studying the Security and Maintainability of MCP Serverscited by 65
- 2023On the Robustness of Code Generation Techniques: An Empirical Study on GitHub Copilotcited by 64
- 2023Can Large Language Models Transform Natural Language Intent into Formal Method Postconditions?cited by 57
- 2024Can LLMs Replace Manual Annotation of Software Engineering Artifacts?cited by 42
- 2024An Empirical Study on Usage and Perceptions of LLMs in a Software Engineering Projectcited by 28
- 2024Search-Based LLMs for Code Optimizationcited by 28
27 more
- 2024AutoDev: Automated AI-Driven Developmentcited by 24
- 2024On the Impacts of Contexts on Repository-Level Code Generationcited by 22
- 2023Evaluating Code Summarization Techniques: A New Metric and an Empirical Characterizationcited by 21
- 2024A Pair Programming Framework for Code Generation via Multi-Plan Exploration and Feedback-Driven Refinementcited by 21
- 2018Categorizing the Content of GitHub README Filescited by 18
- 2023ITER: Iterative Neural Repair for Multi-Location Patchescited by 16
- 2023From Misuse to Mastery: Enhancing Code Generation with Knowledge-Driven AI Chainingcited by 15
- 2024Are LLMs Correctly Integrated into Software Systems?cited by 15
- 2024How to Refactor this Code? An Exploratory Study on Developer-ChatGPT Refactoring Conversationscited by 12
- 2024Can ChatGPT Support Developers? An Empirical Evaluation of Large Language Models for Code Generationcited by 11
- 2025Towards Detecting Prompt Knowledge Gaps for Improved LLM-guided Issue Resolutioncited by 11
- 2025Conversational AI as a Coding Assistant: Understanding Programmers' Interactions with and Expectations from Large Language Models for Codingcited by 11
- 2023Prompt Engineering or Fine-Tuning: An Empirical Assessment of LLMs for Codecited by 9
- 2023On Codex Prompt Engineering for OCL Generation: An Empirical Studycited by 8
- 2024Can Developers Prompt? A Controlled Experiment for Code Documentation Generationcited by 7
- 2024Instruct or Interact? Exploring and Eliciting LLMs' Capability in Code Snippet Adaptation Through Prompt Engineeringcited by 6
- 2025CoDocBench: A Dataset for Code-Documentation Alignment in Software Maintenancecited by 5
- 2025Understanding Prompt Management in GitHub Repositories: A Call for Best Practicescited by 5
- 2021On the Naming of Methods: A Survey of Professional Developerscited by 4
- 2023Wait, wasn't that code here before? Detecting Outdated Software Documentationcited by 3
- 2024Enhancing User Interaction in ChatGPT: Characterizing and Consolidating Multiple Prompts for Issue Resolutioncited by 3
- 2025The Introduction of README and CONTRIBUTING Files in Open Source Software Developmentcited by 3
- 2022An Exploratory Study of Documentation Strategies for Product Features in Popular GitHub Projectscited by 2
- 2018Integrating Runtime Values with Source Code to Facilitate Program Comprehensioncited by 1
- 2022How to Improve Deep Learning for Software Analytics (a case study with code smell detection)cited by 1
- 2022An Exploratory Study on Refactoring Documentation in Issues Handlingcited by 1
- 2022Comments on Comments: Where Code Review and Documentation Meetcited by 1
Built on by
Citing papers per year: 2026: 23.
23 papers.
Sort:
By area:
- 2026Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?cited by 20
- 2026Harness Engineering for Agentic AI Coding Tools: An Exploratory Studycited by 11
- 2026ContextCov: Deriving and Enforcing Executable Constraints from Agent Instruction Filescited by 6
- 2026From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AIcited by 6
- 2026Codified Context: Infrastructure for AI Agents in a Complex Codebasecited by 5
- 2026Getting Better at Working With You: Compiling User Corrections into Runtime Enforcement for Coding Agentscited by 5
- 2026Knowledge Activation: AI Skills as the Institutional Knowledge Primitive for Agentic Software Developmentcited by 4
- 2026A Dataset of Agentic AI Coding Tool Configurationscited by 3
- 2026FAVA: Formal Authorization for Verified Agents with Evidence-Backed Permission Graphscited by 3
- 2026Why Does CLAUDE.md Keep Growing? Catastrophic Remembering in Agentic Codingcited by 3
- 2026The Devil Is in the Interface: Evaluating How Tool Architecture Shapes Coding Agent Behaviorcited by 3
- 2026Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Modelcited by 3
11 more
- 2026From Registry to Repository: How AI Agent Skills Are Written, Adapted, and Maintainedcited by 2
- 2026Rule Taxonomy and Evolution in AI IDEs: A Mining and Survey Studycited by 1
- 2026Probe-and-Refine Tuning of Repository Guidance for Coding Agentscited by 1
- 2026An Exploratory Study of Agent Plans for Agentic AI Coding Tools in Open-Source Softwarecited by 1
- 2026BootstrapAgent: Distilling Repository Setup into Reusable Agent Knowledgecited by 0
- 2026Toward Instructions-as-Code: Understanding the Impact of Instruction Files on Agentic Pull Requestscited by 0
- 2026From Agent Behaviour to Agent-Friendly Documentation: An Empirical Study of How Coding Agents Discover, Read, and Write Technical Documentationcited by 0
- 2026Who Maintains Agent Skills? A Longitudinal Study of Human-Governed, AI-Assisted Skill Maintenancecited by 0
- 2026Scanning the Harness: A Repository Study of Configuration Exposures in AI Coding Agentscited by 0
- 2026Specifying and Maintaining Agentic Workflows: An Empirical Study of GitHub Agentic Workflowscited by 0
- 2026Scientific Agents: Evaluating Profession-Specific System Prompts on Scientific Taskscited by 0
- 2026Specifying and Maintaining Agentic Workflows: An Empirical Study of GitHub Agentic Workflows23 Sep 2026
- 2026Scientific Agents: Evaluating Profession-Specific System Prompts on Scientific Tasks7 Sep 2026
- 2026Scanning the Harness: A Repository Study of Configuration Exposures in AI Coding Agents7 Sep 2026
- 2026Who Maintains Agent Skills? A Longitudinal Study of Human-Governed, AI-Assisted Skill Maintenance4 Sep 2026
- 2026From Agent Behaviour to Agent-Friendly Documentation: An Empirical Study of How Coding Agents Discover, Read, and Write Technical Documentation20 Aug 2026
- 2026Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Model14 Aug 2026
- 2026The Devil Is in the Interface: Evaluating How Tool Architecture Shapes Coding Agent Behavior11 Aug 2026
- 2026Why Does CLAUDE.md Keep Growing? Catastrophic Remembering in Agentic Coding11 Aug 2026
- 2026An Exploratory Study of Agent Plans for Agentic AI Coding Tools in Open-Source Software5 Aug 2026
- 2026FAVA: Formal Authorization for Verified Agents with Evidence-Backed Permission Graphs29 Jul 2026
- 2026From Registry to Repository: How AI Agent Skills Are Written, Adapted, and Maintained1 Jul 2026
- 2026Probe-and-Refine Tuning of Repository Guidance for Coding Agents18 Jun 2026
11 more
- 2026From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI12 Jun 2026
- 2026Toward Instructions-as-Code: Understanding the Impact of Instruction Files on Agentic Pull Requests11 Jun 2026
- 2026Getting Better at Working With You: Compiling User Corrections into Runtime Enforcement for Coding Agents11 Jun 2026
- 2026Rule Taxonomy and Evolution in AI IDEs: A Mining and Survey Study10 Jun 2026
- 2026BootstrapAgent: Distilling Repository Setup into Reusable Agent Knowledge15 May 2026
- 2026A Dataset of Agentic AI Coding Tool Configurations8 May 2026
- 2026Knowledge Activation: AI Skills as the Institutional Knowledge Primitive for Agentic Software Development16 Mar 2026
- 2026ContextCov: Deriving and Enforcing Executable Constraints from Agent Instruction Files28 Feb 2026
- 2026Codified Context: Infrastructure for AI Agents in a Complex Codebase24 Feb 2026
- 2026Harness Engineering for Agentic AI Coding Tools: An Exploratory Study16 Feb 2026
- 2026Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?12 Feb 2026
- 2026Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?cited by 20
- 2026Harness Engineering for Agentic AI Coding Tools: An Exploratory Studycited by 11
- 2026ContextCov: Deriving and Enforcing Executable Constraints from Agent Instruction Filescited by 6
- 2026Codified Context: Infrastructure for AI Agents in a Complex Codebasecited by 5
- 2026Getting Better at Working With You: Compiling User Corrections into Runtime Enforcement for Coding Agentscited by 5
- 2026Knowledge Activation: AI Skills as the Institutional Knowledge Primitive for Agentic Software Developmentcited by 4
- 2026A Dataset of Agentic AI Coding Tool Configurationscited by 3
- 2026Why Does CLAUDE.md Keep Growing? Catastrophic Remembering in Agentic Codingcited by 3
- 2026The Devil Is in the Interface: Evaluating How Tool Architecture Shapes Coding Agent Behaviorcited by 3
- 2026Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Modelcited by 3
- 2026From Registry to Repository: How AI Agent Skills Are Written, Adapted, and Maintainedcited by 2
- 2026Probe-and-Refine Tuning of Repository Guidance for Coding Agentscited by 1
12 most cited of 18 All 18 (JSON)
- 2026Specifying and Maintaining Agentic Workflows: An Empirical Study of GitHub Agentic Workflows23 Sep 2026
- 2026Scanning the Harness: A Repository Study of Configuration Exposures in AI Coding Agents7 Sep 2026
- 2026Who Maintains Agent Skills? A Longitudinal Study of Human-Governed, AI-Assisted Skill Maintenance4 Sep 2026
- 2026From Agent Behaviour to Agent-Friendly Documentation: An Empirical Study of How Coding Agents Discover, Read, and Write Technical Documentation20 Aug 2026
- 2026Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Model14 Aug 2026
- 2026The Devil Is in the Interface: Evaluating How Tool Architecture Shapes Coding Agent Behavior11 Aug 2026
- 2026Why Does CLAUDE.md Keep Growing? Catastrophic Remembering in Agentic Coding11 Aug 2026
- 2026An Exploratory Study of Agent Plans for Agentic AI Coding Tools in Open-Source Software5 Aug 2026
- 2026From Registry to Repository: How AI Agent Skills Are Written, Adapted, and Maintained1 Jul 2026
- 2026Probe-and-Refine Tuning of Repository Guidance for Coding Agents18 Jun 2026
- 2026Toward Instructions-as-Code: Understanding the Impact of Instruction Files on Agentic Pull Requests11 Jun 2026
- 2026Getting Better at Working With You: Compiling User Corrections into Runtime Enforcement for Coding Agents11 Jun 2026
12 newest of 18 All 18 (JSON)
- 2026From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AIcited by 6
- 2026Rule Taxonomy and Evolution in AI IDEs: A Mining and Survey Studycited by 1
- 2026BootstrapAgent: Distilling Repository Setup into Reusable Agent Knowledgecited by 0
- 2026Scientific Agents: Evaluating Profession-Specific System Prompts on Scientific Taskscited by 0
- 2026Scientific Agents: Evaluating Profession-Specific System Prompts on Scientific Tasks7 Sep 2026
- 2026From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI12 Jun 2026
- 2026Rule Taxonomy and Evolution in AI IDEs: A Mining and Survey Study10 Jun 2026
- 2026BootstrapAgent: Distilling Repository Setup into Reusable Agent Knowledge15 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026Do Context Files Help Coding Agents? A Two-Agent Ablation Study on Real Repositories
- 2025Decoding the Configuration of AI Coding Agents: Insights from Claude Code Projects
- 2026Mise en Place for Agentic Coding: Deliberate Preparation as Context Engineering Methodology
- 2026Inside the Scaffold: A Source-Code Taxonomy of Coding Agent Architectures
- 2026When and How Context Rot Appears in Coding Agents: A White-Box Study of Agent Skills in Code Auditing
- 2026AgentSWE: Can Coding Agents Build the Agent You Actually Want?
- 2025Context Engineering for AI Agents in Open-Source Software
- 2026Compact Documentation for Coding Agents: A Benchmark, an Optimizer, and Why It Does Not Transfer