Trace-to-Architecture: Capability Bottleneck Diagnosis for Harness Revision
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Improving an LLM agent requires knowing what to change when it fails. Similar failures can require different fixes, and resolving one problem can expose another. We introduce Trace-to-Architecture (T2A), a framework that uses execution trajectories to determine what to change in an agent's harness and when those changes apply. A development agent diagnoses failures using execution evidence from successful and failed tasks, revises the corresponding harness components, and then checks whether target tasks improve, selected successful tasks remain successful, and the edits are actually triggered. New execution evidence guides subsequent diagnosis and revision. Across ALFWorld, ScienceWorld, and AppWorld, T2A achieves the highest aggregate score for all three evaluated task agents, exceeding their respective strongest baselines by 6.2--30.3 points. We also integrate T2A's diagnosis and validation procedures into Meta-Harness, improving the resulting harnesses' mean ScienceWorld score by 21.6 points from the same starting harness.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 17 of its 25 references that are in the map. First posted on arXiv 12 Oct 2026.
Builds on
17 of 25 references.
Closest ancestors · 6
Same direction, newest first
- 2026SoL-Pi: Recursively Scaling Auto-Research Loops for Efficient Agent Harness17 Sep
- 2026AgentDebugX: An Open-Source Toolkit for Failure Observability, Attribution, and Recovery in LLM Agents21 Jul
- 2026Self-Harness: Harnesses That Improve Themselves8 Jun
- 2026From Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flaws4 Jun
- 2026Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents28 May
- 2026Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses28 Apr
Other references · 11
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
- 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 356
- 2026Meta-Harness: End-to-End Optimization of Model Harnessescited by 309
- 2023Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generationcited by 148
- 2023CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalizationcited by 95
- 2026HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundrycited by 47
- 2026Dream-RSI: Recursive Self-Improvement through Evolving Worldscited by 18
- 2026Self-Evolving World Models for LLM Agent Planningcited by 13
- 2026RRSI: Regularized Recursive Self-Improvement of Agent Harnessescited by 13
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflows
- 2026AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Traces
- 2026TTHE: Test-Time Harness Evolution
- 2026HarnessDev: Can LLMs Create and Evolve Their Own Agent Harness?
- 2026Trace2Skill: Verifier-Guided Skill Evolution for Long-Context EDA Agents
- 2026Vestrum: Improving Agent Harnesses by Adapting Their Verification, Structure and Memory
- 2026Understanding Agent-Reactive Bugs at the Model-Harness Boundary: An Empirical Study of LLM Agent Issue Reports
- 2026VERSE: Verified Self-Evolving Optimizer for Agent Harnesses