mlsanityData to 7 Oct 2026

Meta-Harness: End-to-End Optimization of Model Harnesses

arXiv 2603.28052 ↗First posted 30 Mar 2026cs.AIcited by 248

Not on the map. Why?

Cited by 159 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The performance of large language model (LLM) systems depends not only on model weights, but also on their harness: the code that determines what information to store, retrieve, and present to the model. Yet harnesses are still designed largely by hand, and existing text optimizers are poorly matched to this setting because they compress feedback too aggressively. We introduce Meta-Harness, an outer-loop system that searches over harness code for LLM applications. It uses an agentic proposer that accesses the source code, scores, and execution traces of all prior candidates through a filesystem. On online text classification, Meta-Harness improves over a state-of-the-art context management system by 7.7 points while using 4x fewer context tokens. On retrieval-augmented math reasoning, a single discovered harness improves accuracy on 200 IMO-level problems by 4.7 points on average across five held-out models. On agentic coding, discovered harnesses surpass the best hand-engineered baselines on TerminalBench-2. Together, these results show that richer access to prior experience can enable automated harness engineering.

Key facts. Cited by 248 papers in the map (data through 7 Oct 2026). Builds on 36 of its 54 references that are in the map. First posted on arXiv 30 Mar 2026.

Builds on

36 of 54 references.

References · 36

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  4. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  5. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  6. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  7. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  8. 2023Large Language Models as Optimizerscited by 830
  9. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  10. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 763
  11. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 671
  12. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
24 more
  1. 2016Learning to learn by gradient descent by gradient descentcited by 616
  2. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 515
  3. 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 479
  4. 2025GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningcited by 449
  5. 2020Efficient Intent Detection with Dual Sentence Encoderscited by 419
  6. 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 414
  7. 2024Automated Design of Agentic Systemscited by 360
  8. 2025Agentic Context Engineering: Evolving Contexts for Self-Improving Language Modelscited by 349
  9. 2013Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Textscited by 310
  10. 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 275
  11. 2020The Multilingual Amazon Reviews Corpuscited by 142
  12. 2019Structural Scaffolds for Citation Intent Classification in Scientific Publicationscited by 129
  13. 2022Prompting Is Programming: A Query Language for Large Language Modelscited by 110
  14. 2026Learning to Discover at Test Timecited by 94
  15. 2025MemEvolve: Meta-Evolution of Agent Memory Systemscited by 81
  16. 2025Towards Robust Mathematical Reasoningcited by 67
  17. 2022FiNER: Financial Numeric Entity Recognition for XBRL Taggingcited by 47
  18. 2026AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimizationcited by 46
  19. 2025Feedback Descent: Open-Ended Text Optimization via Pairwise Comparisoncited by 42
  20. 2026Meta Context Engineering via Agentic Skill Evolutioncited by 42
  21. 2024RAR-b: Reasoning as Retrieval Benchmarkcited by 33
  22. 2026Learning to Continually Learn via Meta-learning Agentic Memory Designscited by 16
  23. 2026SWE-Bench Mobile: Can Large Language Model Agents Develop Industry-Level Mobile Applications?cited by 8
  24. 2026Adaptive Retrieval helps Reasoning in LLMs -- but mostly if it's not usedcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 248.

248 papers.

Sort:

By area:

  1. 2026Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnessescited by 119
  2. 2026Self-Harness: Harnesses That Improve Themselvescited by 79
  3. 2026Code as Agent Harnesscited by 40
  4. 2026Rethinking the Evaluation of Harness Evolution for Agentscited by 39
  5. 2026HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundrycited by 37
  6. 2026Continual Harness: Online Adaptation for Self-Improving Foundation Agentscited by 30
  7. 2026Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agentscited by 28
  8. 2026Stop Comparing LLM Agents Without Disclosing the Harnesscited by 25
  9. 2026Adapting the Interface, Not the Model: Runtime Harness Adaptation for Deterministic LLM Agentscited by 22
  10. 2026Recursive Harness Self-Improvementcited by 22
  11. 2026From Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flawscited by 21
  12. 2026Adaptive Auto-Harness: Sustained Self-Improvement for Agentic System Deployment on Open-Ended Task Streamscited by 20
38 more
  1. 2026HarnessForge: Joint Harness and Policy Evolution for Adaptive Agent Systemscited by 20
  2. 2026SIA: Self Improving AI with Harness & Weight Updatescited by 18
  3. 2026RHO: Your Coding Agent is Secretly a Roboticistcited by 18
  4. 2026SkillOpt-Lite: Better and Faster Agent Self-evolution via One Line of Vibecited by 18
  5. 2026Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preferencecited by 17
  6. 2026Co-Harness: Co-Evolving Harnesses and Model Weights for LLM Agentscited by 16
  7. 2026SkillMOO: Multi-Objective Optimization of Agent Skills for Software Engineeringcited by 14
  8. 2026SafeHarness: Lifecycle-Integrated Security Architecture for LLM-based Agent Deploymentcited by 14
  9. 2026MemoHarness: Agent Harnesses That Learn from Experiencecited by 14
  10. 2026HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnessescited by 13
  11. 2026Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectoriescited by 13
  12. 2026ARIS: Autonomous Research via Adversarial Multi-Agent Collaborationcited by 12
  13. 2026Autodata: An agentic data scientist to create high quality synthetic datacited by 12
  14. 2026TTHE: Test-Time Harness Evolutioncited by 12
  15. 2026Evo-Harness: Context-to-Harness Skill Compilation for Self-Evolving Agentscited by 12
  16. 2026Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?cited by 11
  17. 2026HarnessBank: Semantic Gene-Bank Search with Gated Verification for Agent-Harness Self-Evolutioncited by 11
  18. 2026Toward Generalist Autonomous Research via Hypothesis-Tree Refinementcited by 10
  19. 2026Harness-Aware Self-Evolving: Co-Evolving Model Weights, Harness, and Task Solutionscited by 10
  20. 2026JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolutioncited by 10
  21. 2026Agentic Forecasting using Sequential Bayesian Updating of Linguistic Beliefscited by 9
  22. 2026HARBOR: Automated Harness Optimizationcited by 9
  23. 2026Hack-Verifiable Environments: Towards Evaluating Reward Hacking at Scalecited by 9
  24. 2026MOSS: Self-Evolution through Source-Level Rewriting in Autonomous Agent Systemscited by 9
  25. 2026EvoTrainer: Co-Evolving LLM Policies and Training Harnesses for Autonomous Agentic Reinforcement Learningcited by 9
  26. 2026From Question Answering to Task Completion: A Survey on Agent System and Harness Designcited by 9
  27. 2026AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Tracescited by 9
  28. 2026WHALE: A Simple Recipe for Joint Harness-Weight Optimizationcited by 9
  29. 2026Dream-RSI: Recursive Self-Improvement through Evolving Worldscited by 9
  30. 2026Shepherd: Enabling Programmable Meta-Agents via Reversible Agentic Execution Tracescited by 8
  31. 2026The Interplay of Harness Design and Post-Training in LLM Agentscited by 8
  32. 2026Evo-Bench: Can Language Models Improve Agent Harness?cited by 8
  33. 2026Self-Evolving Embodied Agents via Skill-Harness Evolutioncited by 8
  34. 2026LLMs Improving LLMs: Agentic Discovery for Test-Time Scalingcited by 7
  35. 2026Workspace Optimization: How to Train Your Agentcited by 7
  36. 2026The Meta-Agent Challenge: Are Current Agents Capable of Autonomous Agent Development?cited by 7
  37. 2026Better Harnesses, Smaller Models: Building 90% Cheaper Agents via Automated Harness Adaptationcited by 7
  38. 2026Frontis-MA1: Training an AI4AI Model towards Recursive Self-Improvement in Machine Learning Engineeringcited by 7

50 most cited of 248 All 248 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026MESH-Harness: Self-Improving Agent Harnesses via Bandit-Guided Compositional Evolution
  2. 2026LongHarness Bench: Stress-Testing Language Model Harnesses for Long-Context Reasoning
  3. 2025LMR-BENCH: Evaluating LLM Agent's Ability on Reproducing Language Modeling Research
  4. 2026Learning to Control LLM Agent Harnesses with Offline Reinforcement Learning
  5. 2024Large Language Model Evaluation Via Multi AI Agents: Preliminary results
  6. 2026FormulaCode: Evaluating Agentic Optimization on Large Codebases
  7. 2026The Empire, Long Divided, Must Unite: Architectural Convergence in Three LLM Agent Harnesses
  8. 2024Designing Algorithms Empowered by Language Models: An Analytical Framework, Case Studies, and Insights