Better Harnesses, Smaller Models: Building 90% Cheaper Agents via Automated Harness Adaptation
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Frontier LLM agents are automating many business tasks, but their high inference cost makes large-scale deployment unsustainable. Small language models (SLMs) offer a cheaper alternative, yet they typically fall short when swapped into a harness designed for a frontier LLM. We show that for many routine business tasks, SLM agents can match LLM performance at 90% lower cost, when paired with an adapted harness that can be automatically discovered by a meta agent. The key insight is that much of the task difficulty is shared across instances and can be lifted from the model into the harness via tailored instructions, tools, and orchestration loops. To study this systematically, we create a framework that maps agent failure modes to harness adaptation strategies, and build a harness optimizer that automatically discovers effective adaptations from failure trajectories. Across seven business-oriented agentic tasks and three SLM families, we found optimized harnesses significantly improve performance on 16 of 21 task-SLM pairs, with seven pairs closing the SLM-LLM performance gap and the best SLM agent recovering 89.7% of LLM performance at 4% of the cost. Our analysis further shows that adaptation works best for tasks with more repetitive workflows and for SLMs with sufficient base capabilities. Together, these results suggest that harness adaptation can expand the practical deployment range of SLM agents in routine business tasks.
Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 41 of its 59 references that are in the map. First posted on arXiv 9 Jul 2026.
Builds on
41 of 59 references.
Closest ancestors · 8
Same direction, newest first
- 2026Adapting the Interface, Not the Model: Runtime Harness Adaptation for Deterministic LLM Agents21 May
- 2026Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?12 Feb
- 2025BOAD: Discovering Hierarchical Software Engineering Agents via Bandit Optimization29 Dec
- 2025Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly?17 Nov
- 2025The OpenHands Software Agent SDK: A Composable and Extensible Foundation for Production Agents5 Nov
- 2025Where LLM Agents Fail and How They can Learn From Failures29 Sep
- 2025WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratch6 May
- 2025Why Do Multi-Agent LLM Systems Fail?17 Mar
Other references · 33
- 2025Qwen3 Technical Reportcited by 8,820
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
- 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
- 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
- 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
- 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
- 2025GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningcited by 542
- 2026Kimi K2.5: Visual Agentic Intelligencecited by 501
- 2024Automated Design of Agentic Systemscited by 407
- 2025Agentic Context Engineering: Evolving Contexts for Self-Improving Language Modelscited by 398
21 more
- 2024Measuring short-form factuality in large language modelscited by 373
- 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 356
- 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Taskscited by 326
- 2026Meta-Harness: End-to-End Optimization of Model Harnessescited by 309
- 2025Small Language Models are the Future of Agentic AIcited by 270
- 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Taskscited by 232
- 2025Multi-agent Architecture Search via Agentic Supernetcited by 179
- 2025GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Taskscited by 158
- 2024AgentSquare: Automatic LLM Agent Search in Modular Design Spacecited by 155
- 2025Towards a Science of Scaling Agent Systemscited by 138
- 2025ACON: Optimizing Context Compression for Long-horizon LLM Agentscited by 136
- 2024AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agentscited by 114
- 2025Multi-Agent Design: Optimizing Agents with Better Prompts and Topologiescited by 113
- 2025Inducing Programmatic Skills for Agentic Taskscited by 111
- 2025Recursive Language Modelscited by 104
- 2026MCP-Atlas: A Large-Scale Benchmark for Tool-Use Competency with Real MCP Serverscited by 53
- 2025What Prompts Don't Say: Understanding and Managing Underspecification in LLM Promptscited by 32
- 2025RefactorBench: Evaluating Stateful Reasoning in Language Agents Through Codecited by 30
- 2026LOCA-bench: Benchmarking Language Agents Under Controllable and Extreme Context Growthcited by 14
- 2025Don't Adapt Small Language Models for Tools; Adapt Tool Schemas to the Modelscited by 6
- 2026EffGen: Enabling Small Language Models as Capable Autonomous Agentscited by 1
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026RRSI: Regularized Recursive Self-Improvement of Agent Harnessescited by 13
- 2026Beyond Final Scores: A Systematic Evaluation of Agents for Long-Horizon AI Research and Developmentcited by 4
- 2026Co-Evolving Harnesses and Models: On-Policy Correction Helps Weaker Models Catch Up Where Imitation Failscited by 4
- 2026StateM: Reaching 95.3% Raw Accuracy, or a 15 Frontier Run, on Terminal-Bench 2.1 via Harness Scalingcited by 1
- 2026LiteEvo: Automated, Cost-Efficient Harness Evolution for Generalization to Unseen Taskscited by 1
- 2026CoTrace: Data Recipes for Training Terminal Agents with Harness-Model Co-Evolutioncited by 1
- 2026Harness Compilation: Which Decisions Should a Small Vision-Language Model Keep?cited by 0
- 2026Harness Compilation: Which Decisions Should a Small Vision-Language Model Keep?8 Oct 2026
- 2026CoTrace: Data Recipes for Training Terminal Agents with Harness-Model Co-Evolution7 Oct 2026
- 2026LiteEvo: Automated, Cost-Efficient Harness Evolution for Generalization to Unseen Tasks27 Sep 2026
- 2026RRSI: Regularized Recursive Self-Improvement of Agent Harnesses21 Sep 2026
- 2026Co-Evolving Harnesses and Models: On-Policy Correction Helps Weaker Models Catch Up Where Imitation Fails8 Sep 2026
- 2026StateM: Reaching 95.3% Raw Accuracy, or a 15 Frontier Run, on Terminal-Bench 2.1 via Harness Scaling15 Aug 2026
- 2026Beyond Final Scores: A Systematic Evaluation of Agents for Long-Horizon AI Research and Development13 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2026Self-Harness: Harnesses That Improve Themselves
- 2026HarnessOpt-Bench: Evaluating LLMs at Harness Optimization
- 2026Ecdysis: Efficient and Effective Training of Runtime Harnesses for LLM Agents
- 2026Stop Comparing LLM Agents Without Disclosing the Harness
- 2026Where Does Harness-Optimization Value Live? Localized Gains and the Budget-Splitting Trap in Self-Evolving LLM Agents
- 2026Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents
- 2026HarnessForge: Joint Harness and Policy Evolution for Adaptive Agent Systems
- 2026Grow the Harness, Not the Context: From Strategy-Free Scaffolds to Reusable Specialist Agents