# Meta-Harness: End-to-End Optimization of Model Harnesses

arXiv: https://arxiv.org/abs/2603.28052 · HTML: https://mlsanity.com/paper/2603.28052/

## Key facts

- Cited by 309 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 54 references that are in the map.
- First posted on arXiv 30 Mar 2026.

## Builds on (most cited first, 30 of 36)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions](https://mlsanity.com/paper/2212.10509/) (2022)
- [AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://mlsanity.com/paper/2506.13131/) (2025)
- [Large Language Models as Optimizers](https://mlsanity.com/paper/2309.03409/) (2023)
- [DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines](https://mlsanity.com/paper/2310.03714/) (2023)
- [What learning algorithm is in-context learning? Investigations with linear models](https://mlsanity.com/paper/2211.15661/) (2022)
- [Automatic Prompt Optimization with "Gradient Descent" and Beam Search](https://mlsanity.com/paper/2305.03495/) (2023)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces](https://mlsanity.com/paper/2601.11868/) (2026)
- [GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning](https://mlsanity.com/paper/2507.19457/) (2025)
- [GoEmotions: A Dataset of Fine-Grained Emotions](https://mlsanity.com/paper/2005.00547/) (2020)
- [Efficient Intent Detection with Dual Sentence Encoders](https://mlsanity.com/paper/2003.04807/) (2020)
- [TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification](https://mlsanity.com/paper/2010.12421/) (2020)
- [Automated Design of Agentic Systems](https://mlsanity.com/paper/2408.08435/) (2024)
- [Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models](https://mlsanity.com/paper/2510.04618/) (2025)
- [Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Texts](https://mlsanity.com/paper/1307.5336/) (2013)
- [MathArena: Evaluating LLMs on Uncontaminated Math Competitions](https://mlsanity.com/paper/2505.23281/) (2025)
- [The Multilingual Amazon Reviews Corpus](https://mlsanity.com/paper/2010.02573/) (2020)
- [Structural Scaffolds for Citation Intent Classification in Scientific Publications](https://mlsanity.com/paper/1904.01608/) (2019)
- [Learning to Discover at Test Time](https://mlsanity.com/paper/2601.16175/) (2026)
- [Prompting Is Programming: A Query Language for Large Language Models](https://mlsanity.com/paper/2212.06094/) (2022)
- [MemEvolve: Meta-Evolution of Agent Memory Systems](https://mlsanity.com/paper/2512.18746/) (2025)
- [Towards Robust Mathematical Reasoning](https://mlsanity.com/paper/2511.01846/) (2025)
- [AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization](https://mlsanity.com/paper/2602.20133/) (2026)
- [FiNER: Financial Numeric Entity Recognition for XBRL Tagging](https://mlsanity.com/paper/2203.06482/) (2022)

## Built on by (most cited first, 30 of 309)

- [Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses](https://mlsanity.com/paper/2604.25850/) (2026)
- [Self-Harness: Harnesses That Improve Themselves](https://mlsanity.com/paper/2606.09498/) (2026)
- [Rethinking the Evaluation of Harness Evolution for Agents](https://mlsanity.com/paper/2607.12227/) (2026)
- [Code as Agent Harness](https://mlsanity.com/paper/2605.18747/) (2026)
- [HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundry](https://mlsanity.com/paper/2606.14249/) (2026)
- [Continual Harness: Online Adaptation for Self-Improving Foundation Agents](https://mlsanity.com/paper/2605.09998/) (2026)
- [Stop Comparing LLM Agents Without Disclosing the Harness](https://mlsanity.com/paper/2605.23950/) (2026)
- [Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents](https://mlsanity.com/paper/2605.30621/) (2026)
- [Recursive Harness Self-Improvement](https://mlsanity.com/paper/2607.15524/) (2026)
- [From Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flaws](https://mlsanity.com/paper/2606.06324/) (2026)
- [Adapting the Interface, Not the Model: Runtime Harness Adaptation for Deterministic LLM Agents](https://mlsanity.com/paper/2605.22166/) (2026)
- [Adaptive Auto-Harness: Sustained Self-Improvement for Agentic System Deployment on Open-Ended Task Streams](https://mlsanity.com/paper/2606.01770/) (2026)
- [Toward Generalist Autonomous Research via Hypothesis-Tree Refinement](https://mlsanity.com/paper/2606.11926/) (2026)
- [SIA: Self Improving AI with Harness & Weight Updates](https://mlsanity.com/paper/2605.27276/) (2026)
- [HarnessForge: Joint Harness and Policy Evolution for Adaptive Agent Systems](https://mlsanity.com/paper/2606.01779/) (2026)
- [RHO: Your Coding Agent is Secretly a Roboticist](https://mlsanity.com/paper/2606.16458/) (2026)
- [SkillOpt-Lite: Better and Faster Agent Self-evolution via One Line of Vibe](https://mlsanity.com/paper/2607.03451/) (2026)
- [Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference](https://mlsanity.com/paper/2606.05922/) (2026)
- [Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?](https://mlsanity.com/paper/2607.09996/) (2026)
- [Co-Harness: Co-Evolving Harnesses and Model Weights for LLM Agents](https://mlsanity.com/paper/2607.22688/) (2026)
- [Dream-RSI: Recursive Self-Improvement through Evolving Worlds](https://mlsanity.com/paper/2609.14858/) (2026)
- [HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnesses](https://mlsanity.com/paper/2608.01918/) (2026)
- [ARIS: Autonomous Research via Adversarial Multi-Agent Collaboration](https://mlsanity.com/paper/2605.03042/) (2026)
- [Autodata: An agentic data scientist to create high quality synthetic data](https://mlsanity.com/paper/2606.25996/) (2026)
- [TTHE: Test-Time Harness Evolution](https://mlsanity.com/paper/2607.08124/) (2026)
- [MemoHarness: Agent Harnesses That Learn from Experience](https://mlsanity.com/paper/2607.14159/) (2026)
- [Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectories](https://mlsanity.com/paper/2608.02276/) (2026)
- [Evo-Harness: Context-to-Harness Skill Compilation for Self-Evolving Agents](https://mlsanity.com/paper/2608.15071/) (2026)
- [JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolution](https://mlsanity.com/paper/2608.25593/) (2026)
- [SkillMOO: Multi-Objective Optimization of Agent Skills for Software Engineering](https://mlsanity.com/paper/2604.09297/) (2026)

## Abstract (arXiv)

The performance of large language model (LLM) systems depends not only on model weights, but also on their harness: the code that determines what information to store, retrieve, and present to the model. Yet harnesses are still designed largely by hand, and existing text optimizers are poorly matched to this setting because they compress feedback too aggressively. We introduce Meta-Harness, an outer-loop system that searches over harness code for LLM applications. It uses an agentic proposer that accesses the source code, scores, and execution traces of all prior candidates through a filesystem. On online text classification, Meta-Harness improves over a state-of-the-art context management system by 7.7 points while using 4x fewer context tokens. On retrieval-augmented math reasoning, a single discovered harness improves accuracy on 200 IMO-level problems by 4.7 points on average across five held-out models. On agentic coding, discovered harnesses surpass the best hand-engineered baselines on TerminalBench-2. Together, these results show that richer access to prior experience can enable automated harness engineering.