Harness Handbook: Making Evolving Agent Harnesses Readable,Navigable, and Editable
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
The capability of a modern AI agent depends not only on its foundation model but also on its harness, which constructs prompts, manages state, invokes tools, and coordinates execution. As models, APIs, environments, and requirements evolve, the harness must be continually modified. Before such a change can be made, a developer or coding agent must identify all code locations that implement the target behavior. This is difficult because production harnesses are large, tightly coupled, and behaviorally distributed, while modification requests describe what the system should do and repositories are organized by files and modules. Code search, repository indexing, and long-context processing ease inspection, but still leave this behavior-to-code mapping to be recovered by hand. Behavior localization is therefore a central bottleneck in harness evolution. We introduce the Harness Handbook, a behavior-centric representation synthesized automatically from a harness codebase via static analysis and LLM-assisted structuring, linking each behavior to its corresponding source. We also introduce Behavior-Guided Progressive Disclosure (BGPD), which guides agents from high-level behaviors to relevant implementation details and verifies candidate locations against the current source. On diverse modification requests from two open-source harnesses, Handbook-Assisted planning improves behavior localization and edit-plan quality while using fewer planner tokens, with the largest gains on scattered sites, rarely executed paths, and cross-module interactions. Evolving complex agentic systems thus depends not only on generating edits, but also on determining where those edits should be made.
Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 28 of its 36 references that are in the map. First posted on arXiv 14 Jul 2026.
Builds on
28 of 36 references.
Closest ancestors · 12
Same direction, newest first
- 2026SHERLOC: Structured Diagnostic Localization for Code Repair Agents23 Jun
- 2026Code Isn't Memory: A Structural Codebase Index Inside a Coding Agent21 Jun
- 2026SWE-Explore: Benchmarking How Coding Agents Explore Repositories5 Jun
- 2026From Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flaws4 Jun
- 2026HarnessForge: Joint Harness and Policy Evolution for Adaptive Agent Systems1 Jun
- 2026Code as Agent Harness18 May
- 2026AI Harness Engineering: A Runtime Substrate for Foundation-Model Software Agents13 May
- 2026Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses28 Apr
- 2026Repository Intelligence Graph: Deterministic Architectural Map for LLM Code Assistants15 Jan
- 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agents23 Jul
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering6 May
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation16 Aug
Other references · 16
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
- 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 515
- 2026Natural-Language Agent Harnessescited by 54
- 2026HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundrycited by 37
- 2025Towards Agentic Recommender Systems in the Era of Multimodal Large Language Modelscited by 25
- 2025Improving Code Localization with Repository Memorycited by 19
- 2025Nex-N1: Agentic Models Trained via a Unified Ecosystem for Large-Scale Environment Constructioncited by 17
4 more
- 2026Coding Agents are Effective Long-Context Processorscited by 11
- 2025Federated In-Context Learning: Iterative Refinement for Improved Answer Qualitycited by 7
- 2026SemaClaw: A Step Towards General-Purpose Personal AI Agents through Harness Engineeringcited by 4
- 2026FERA: Uncertainty-Aware Federated Reasoning for Large Language Modelscited by 1
Built on by
Citing papers per year: 2026: 12.
12 papers.
- 2026Recursive Synthesis for Long-Horizon Terminal Taskscited by 8
- 2026RRSI: Regularized Recursive Self-Improvement of Agent Harnessescited by 3
- 2026ClawGym II: Exploring Black-Box RL on Agent Harnesscited by 2
- 2026Ecdysis: Efficient and Effective Training of Runtime Harnesses for LLM Agentscited by 2
- 2026Agent Harness Distillation: Inference-Time Harness Extraction and Exploitation in Autonomous Multi-Agent Systemscited by 1
- 2026Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligencecited by 1
- 2026T1: Terminal Agent Reinforcement Learning for Long-Horizon Taskscited by 1
- 2026More Programs or More Rolls? Separating Coverage from Specialization in LLM Harnessescited by 1
- 2026Continual Learning in Transitioncited by 0
- 2026ADIAS: Automated Design of Interactive Agentic Systemscited by 0
- 2026From Prompt to Harness: Coderlet from Scratchcited by 0
- 2026A Contract-Centered Architecture for Scalable and Manageable Agentic Runtimescited by 0
- 2026More Programs or More Rolls? Separating Coverage from Specialization in LLM Harnesses26 Sep 2026
- 2026RRSI: Regularized Recursive Self-Improvement of Agent Harnesses21 Sep 2026
- 2026Ecdysis: Efficient and Effective Training of Runtime Harnesses for LLM Agents10 Sep 2026
- 2026T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks10 Sep 2026
- 2026A Contract-Centered Architecture for Scalable and Manageable Agentic Runtimes27 Aug 2026
- 2026Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence21 Aug 2026
- 2026ClawGym II: Exploring Black-Box RL on Agent Harness17 Aug 2026
- 2026From Prompt to Harness: Coderlet from Scratch10 Aug 2026
- 2026Continual Learning in Transition6 Aug 2026
- 2026Recursive Synthesis for Long-Horizon Terminal Tasks5 Aug 2026
- 2026ADIAS: Automated Design of Interactive Agentic Systems3 Aug 2026
- 2026Agent Harness Distillation: Inference-Time Harness Extraction and Exploitation in Autonomous Multi-Agent Systems30 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2026Verify Smarter, Evolve Further: Efficient Harness Evolution through Behavior-Aware Verification
- 2026HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnesses
- 2026Harness-Zero: Harness Distillation via Agent-as-Harness
- 2026Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectories
- 2026Dissecting model behavior through agent trajectories
- 2026Co-Evolving Harnesses and Models: On-Policy Correction Helps Weaker Models Catch Up Where Imitation Fails
- 2026ModularRSI: Modular and Generalizable Recursive Harness Self-Improvement
- 2026HarnessPAI: An Evolving Harness for Physical AI