mlsanityData to 14 Oct 2026

Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems

arXiv 2609.00006 ↗First posted 15 Jul 2026cs.SE cs.MAcited by 6

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

An agent is a model plus a harness -- the runtime that couples an LLM to the world through a loop, tools, context management, safety controls, orchestration, and extension surfaces. Harness engineering, named as a discipline in early 2026, is the design and evolution of that runtime. This paper gives the young discipline its most comprehensive empirical foundation to date: a source-code anatomy of eleven production coding harnesses (Claude Code, Codex CLI, Gemini CLI, Mistral Vibe, OpenHands, Aider, Mini-SWE-Agent, Hermes, Pi, OpenCode, OpenClaw), plus Omnigent, the first meta-harness, analyzed as a contrast point. We define what a harness is, map its seven canonical subsystems with the minimal and maximal implementation of each, and dissect all eleven systems along those subsystems. The audit yields 13 cross-cutting observations and a catalog of 29 recurring design patterns. Two absences survive a threefold corpus expansion: across roughly four million lines of Python, TypeScript, and Rust, no agent runtime imports a general-purpose agentic framework, and none retrieves code with vector embeddings; the field runs on hand-rolled async loops and deterministic retrieval. SKILL.md skills lead MCP in adoption (9/11 vs. 8/11), and ACP ships in six systems with a new third role: harness hosting. Because the original eight systems were re-pinned rather than replaced, the study also contains a controlled longitudinal sample -- the same harnesses source-diffed across one quarter -- showing convergence becoming imitation and behavioral policy migrating from prompt prose to configuration. These threads converge on the paper's thesis: in the first half of 2026 the coding harness completed a turn from tool to platform. The paper closes with 18 design recommendations and a 90-line minimum-viable-harness scaffold.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 55 of its 92 references that are in the map. First posted on arXiv 15 Jul 2026.

Builds on

55 of 92 references.

Closest ancestors · 17

Same direction, newest first

  1. 2026What makes a harness a harness: necessary and sufficient conditions for an agent harness8 Jun
  2. 2026Code as Agent Harness18 May
  3. 2026Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses28 Apr
  4. 2026HARBOR: Automated Harness Optimization22 Apr
  5. 2026Inside the Scaffold: A Source-Code Taxonomy of Coding Agent Architectures3 Apr
  6. 2026Agentic Much? Adoption of Coding Agents on GitHub26 Jan
  7. 2025The Rise of AI Teammates in Software Engineering (SE) 3.0: How Autonomous Coding Agents Are Reshaping Software Engineering20 Jul
  8. 2024Lingma SWE-GPT: An Open Development-Process-Centric Language Model for Automated Software Improvement1 Nov
  9. 2024SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement26 Oct
  10. 2024Large Language Model-Based Agents for Software Engineering: A Survey4 Sep
  11. 2024Diversity Empowers Intelligence: Integrating Expertise of Software Engineering Agents13 Aug
  12. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agents23 Jul
5 more
  1. 2024Agentless: Demystifying LLM-based Software Engineering Agents1 Jul
  2. 2024MASAI: Modular Architecture for Software-engineering AI Agents17 Jun
  3. 2024MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution26 Mar
  4. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation16 Aug
  5. 2023ChatDev: Communicative Agents for Software Development16 Jul

Other references · 38

  1. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  2. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  3. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  4. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  5. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  6. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  7. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  8. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
  9. 2023Teaching Large Language Models to Self-Debugcited by 988
  10. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
  11. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  12. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 769
26 more
  1. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
  2. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 574
  3. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 560
  4. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 372
  5. 2023RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generationcited by 367
  6. 2023RepoBench: Benchmarking Repository-Level Code Auto-Completion Systemscited by 353
  7. 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agentscited by 288
  8. 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 287
  9. 2024AutoCodeRover: Autonomous Program Improvementcited by 278
  10. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 268
  11. 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Taskscited by 232
  12. 2024Chain of Agents: Large Language Models Collaborating on Long-Context Taskscited by 206
  13. 2023CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completioncited by 183
  14. 2024AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agentscited by 145
  15. 2024AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Callscited by 124
  16. 2026Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forwardcited by 113
  17. 2024CodeR: Issue Resolving with Multi-Agent and Task Graphscited by 91
  18. 2024CodeRAG-Bench: Can Retrieval Augment Code Generation?cited by 91
  19. 2024AgentInstruct: Toward Generative Teaching with Agentic Flowscited by 89
  20. 2024ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stagescited by 82
  21. 2024RepoAgent: An LLM-Powered Open-Source Framework for Repository-level Code Documentation Generationcited by 70
  22. 2024Long Code Arena: a Set of Benchmarks for Long-Context Code Modelscited by 50
  23. 2026SkillProbe: Security Auditing for Emerging Agent Skill Marketplaces via Multi-Agent Collaborationcited by 21
  24. 2025A Measurement Study of Model Context Protocol Ecosystemcited by 13
  25. 2025Build Agent Advocates, Not Platform Agentscited by 8
  26. 2026Skilldex: A Package Manager and Registry for Agent Skill Packages with Hierarchical Scope-Based Distributioncited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026An Empirical Study of Harness Design for Coding Agents
  2. 2026The Empire, Long Divided, Must Unite: Architectural Convergence in Three LLM Agent Harnesses
  3. 2026Don't Blame the Large Language Model: How Agent Harness Evolution Shapes Coding Agent Quality
  4. 2026Harness Engineering as Categorical Architecture
  5. 2026Composing Task-specific Agent Harnesses at Test Time with Reusable Primitives
  6. 2026Harness or Model? Isolating the Harness Effect in Agentic Coding with a Contamination-Controlled Private Suite
  7. 2026One Recipe, Many Harnesses: What Self-Evolution Encodes Across Languages and Models ↗
  8. 2026The Devil Is in the Interface: Evaluating How Tool Architecture Shapes Coding Agent Behavior