Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
An agent is a model plus a harness -- the runtime that couples an LLM to the world through a loop, tools, context management, safety controls, orchestration, and extension surfaces. Harness engineering, named as a discipline in early 2026, is the design and evolution of that runtime. This paper gives the young discipline its most comprehensive empirical foundation to date: a source-code anatomy of eleven production coding harnesses (Claude Code, Codex CLI, Gemini CLI, Mistral Vibe, OpenHands, Aider, Mini-SWE-Agent, Hermes, Pi, OpenCode, OpenClaw), plus Omnigent, the first meta-harness, analyzed as a contrast point. We define what a harness is, map its seven canonical subsystems with the minimal and maximal implementation of each, and dissect all eleven systems along those subsystems. The audit yields 13 cross-cutting observations and a catalog of 29 recurring design patterns. Two absences survive a threefold corpus expansion: across roughly four million lines of Python, TypeScript, and Rust, no agent runtime imports a general-purpose agentic framework, and none retrieves code with vector embeddings; the field runs on hand-rolled async loops and deterministic retrieval. SKILL.md skills lead MCP in adoption (9/11 vs. 8/11), and ACP ships in six systems with a new third role: harness hosting. Because the original eight systems were re-pinned rather than replaced, the study also contains a controlled longitudinal sample -- the same harnesses source-diffed across one quarter -- showing convergence becoming imitation and behavioral policy migrating from prompt prose to configuration. These threads converge on the paper's thesis: in the first half of 2026 the coding harness completed a turn from tool to platform. The paper closes with 18 design recommendations and a 90-line minimum-viable-harness scaffold.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 55 of its 92 references that are in the map. First posted on arXiv 15 Jul 2026.
Builds on
55 of 92 references.
Closest ancestors · 17
Same direction, newest first
- 2026What makes a harness a harness: necessary and sufficient conditions for an agent harness8 Jun
- 2026Code as Agent Harness18 May
- 2026Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses28 Apr
- 2026HARBOR: Automated Harness Optimization22 Apr
- 2026Inside the Scaffold: A Source-Code Taxonomy of Coding Agent Architectures3 Apr
- 2026Agentic Much? Adoption of Coding Agents on GitHub26 Jan
- 2025The Rise of AI Teammates in Software Engineering (SE) 3.0: How Autonomous Coding Agents Are Reshaping Software Engineering20 Jul
- 2024Lingma SWE-GPT: An Open Development-Process-Centric Language Model for Automated Software Improvement1 Nov
- 2024SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement26 Oct
- 2024Large Language Model-Based Agents for Software Engineering: A Survey4 Sep
- 2024Diversity Empowers Intelligence: Integrating Expertise of Software Engineering Agents13 Aug
- 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agents23 Jul
5 more
- 2024Agentless: Demystifying LLM-based Software Engineering Agents1 Jul
- 2024MASAI: Modular Architecture for Software-engineering AI Agents17 Jun
- 2024MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution26 Mar
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation16 Aug
- 2023ChatDev: Communicative Agents for Software Development16 Jul
Other references · 38
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
- 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
- 2023Teaching Large Language Models to Self-Debugcited by 988
- 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
- 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
- 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 769
26 more
- 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
- 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 574
- 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 560
- 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 372
- 2023RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generationcited by 367
- 2023RepoBench: Benchmarking Repository-Level Code Auto-Completion Systemscited by 353
- 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agentscited by 288
- 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 287
- 2024AutoCodeRover: Autonomous Program Improvementcited by 278
- 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 268
- 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Taskscited by 232
- 2024Chain of Agents: Large Language Models Collaborating on Long-Context Taskscited by 206
- 2023CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completioncited by 183
- 2024AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agentscited by 145
- 2024AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Callscited by 124
- 2026Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forwardcited by 113
- 2024CodeR: Issue Resolving with Multi-Agent and Task Graphscited by 91
- 2024CodeRAG-Bench: Can Retrieval Augment Code Generation?cited by 91
- 2024AgentInstruct: Toward Generative Teaching with Agentic Flowscited by 89
- 2024ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stagescited by 82
- 2024RepoAgent: An LLM-Powered Open-Source Framework for Repository-level Code Documentation Generationcited by 70
- 2024Long Code Arena: a Set of Benchmarks for Long-Context Code Modelscited by 50
- 2026SkillProbe: Security Auditing for Emerging Agent Skill Marketplaces via Multi-Agent Collaborationcited by 21
- 2025A Measurement Study of Model Context Protocol Ecosystemcited by 13
- 2025Build Agent Advocates, Not Platform Agentscited by 8
- 2026Skilldex: A Package Manager and Registry for Agent Skill Packages with Hierarchical Scope-Based Distributioncited by 5
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026Where Does Exactly-Once Live? Model, Harness, and Tool-Contract Effects on Duplicate Side Effects in LLM Agentscited by 3
- 2026Trained Agentic Context Managementcited by 0
- 2026Do Tool Calls Execute as Intended? Measuring and Repairing Intent-Execution Correspondence in LLM Agentscited by 0
- 2026Words to World: Benchmarking LLMs in Interactive Physical Environmentscited by 0
- 2026L3M: A Verifiable Harness and Runtime for Self-Extending and Self-Improving Agentscited by 0
- 2026Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agentscited by 0
- 2026Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agents14 Oct 2026
- 2026L3M: A Verifiable Harness and Runtime for Self-Extending and Self-Improving Agents14 Oct 2026
- 2026Words to World: Benchmarking LLMs in Interactive Physical Environments10 Oct 2026
- 2026Do Tool Calls Execute as Intended? Measuring and Repairing Intent-Execution Correspondence in LLM Agents3 Oct 2026
- 2026Trained Agentic Context Management1 Oct 2026
- 2026Where Does Exactly-Once Live? Model, Harness, and Tool-Contract Effects on Duplicate Side Effects in LLM Agents24 Sep 2026
Similar, not cited
Close in wording, no citation either way
- 2026An Empirical Study of Harness Design for Coding Agents
- 2026The Empire, Long Divided, Must Unite: Architectural Convergence in Three LLM Agent Harnesses
- 2026Don't Blame the Large Language Model: How Agent Harness Evolution Shapes Coding Agent Quality
- 2026Harness Engineering as Categorical Architecture
- 2026Composing Task-specific Agent Harnesses at Test Time with Reusable Primitives
- 2026Harness or Model? Isolating the Harness Effect in Agentic Coding with a Contamination-Controlled Private Suite
- 2026One Recipe, Many Harnesses: What Self-Evolution Encodes Across Languages and Models ↗
- 2026The Devil Is in the Interface: Evaluating How Tool Architecture Shapes Coding Agent Behavior