mlsanityData to 7 Oct 2026

Harness Engineering for Agentic AI Coding Tools: An Exploratory Study

arXiv 2602.14690 ↗First posted 16 Feb 2026cs.SEcited by 10

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Agentic AI coding tools increasingly automate software development tasks. Developers can configure these tools through versioned repository-level artifacts such as Markdown and JSON files. We present a systematic analysis of configuration mechanisms for agentic AI coding tools, covering Claude Code, GitHub Copilot, Cursor, Gemini, and Codex. We identify eight configuration mechanisms spanning from static context to executable and external integrations and, in an empirical study of 2,853 GitHub repositories, examine whether and how they are adopted, with a detailed analysis of Context Files, Skills, and Subagents. First, Context Files dominate the configuration landscape and are often the sole mechanism in a repository, with AGENTS.md emerging as an interoperable standard across tools. Second, few repositories adopt advanced mechanisms such as Skills and Subagents. Skills predominantly rely on static instructions rather than executable scripts. Third, distinct configuration practices are forming around different tools, with Claude Code users employing the broadest range of mechanisms. These findings establish an empirical baseline for understanding how developers configure agentic tools, suggest that AGENTS.md serves as a natural starting point, and motivate longitudinal and experimental research on how configuration strategies evolve and affect agent performance.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 13 of its 32 references that are in the map. First posted on arXiv 16 Feb 2026.

Builds on

13 of 32 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026On the Impact of AGENTS.md Files on the Efficiency of AI Coding Agents28 Jan
  2. 2026Agentic Much? Adoption of Coding Agents on GitHub26 Jan
  3. 2025Agent READMEs: An Empirical Study of Context Files for Agentic Coding17 Nov
  4. 2025Decoding the Configuration of AI Coding Agents: Insights from Claude Code Projects12 Nov
  5. 2025Speed at the Cost of Quality: How Cursor AI Increases Short-Term Velocity and Long-Term Complexity in Open-Source Projects6 Nov

Other references · 8

  1. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 428
  2. 2024Using AI-Based Coding Assistants in Practice: State of Affairs, Perceptions, and Ways Forwardcited by 44
  3. 2026Meta Context Engineering via Agentic Skill Evolutioncited by 42
  4. 2021Sampling Projects in GitHub for MSR Studiescited by 20
  5. 2025Context Engineering for AI Agents in Open-Source Softwarecited by 14
  6. 2025Beyond the Prompt: An Empirical Study of Cursor Rulescited by 7
  7. 2026Structured Context Engineering for File-Native Agentic Systems: Evaluating Schema Accuracy, Format Effectiveness, and Multi-File Navigation at Scalecited by 4
  8. 2025Prompts as Software Engineering Artifacts: A Research Agenda and Preliminary Findingscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 10.

10 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026A Dataset of Agentic AI Coding Tool Configurations
  2. 2026An Exploratory Study of Agent Plans for Agentic AI Coding Tools in Open-Source Software
  3. 2026Scanning the Harness: A Repository Study of Configuration Exposures in AI Coding Agents
  4. 2025On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub
  5. 2025On the Use of Agentic Coding Manifests: An Empirical Study of Claude Code
  6. 2026Code as Agent Harness
  7. 2025Agentic Refactoring: An Empirical Study of AI Coding Agents
  8. 2026Specifying and Maintaining Agentic Workflows: An Empirical Study of GitHub Agentic Workflows