# HarnessEvolve: Learning from Reference Trajectories for Reliable Agent Self-Evolution

arXiv: https://arxiv.org/abs/2609.00829 · HTML: https://mlsanity.com/paper/2609.00829/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 33 references that are in the map.
- First posted on arXiv 1 Sep 2026.

## Builds on (most cited first, 30 of 30)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [Measuring and Narrowing the Compositionality Gap in Language Models](https://mlsanity.com/paper/2210.03350/) (2022)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [Graph of Thoughts: Solving Elaborate Problems with Large Language Models](https://mlsanity.com/paper/2308.09687/) (2023)
- [ExpeL: LLM Agents Are Experiential Learners](https://mlsanity.com/paper/2308.10144/) (2023)
- [Large Language Models as Optimizers](https://mlsanity.com/paper/2309.03409/) (2023)
- [Large-Scale Long-Tailed Recognition in an Open World](https://mlsanity.com/paper/1904.05160/) (2019)
- [Automatic Prompt Optimization with "Gradient Descent" and Beam Search](https://mlsanity.com/paper/2305.03495/) (2023)
- [GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning](https://mlsanity.com/paper/2507.19457/) (2025)
- [Does Learning Require Memorization? A Short Tale about a Long Tail](https://mlsanity.com/paper/1906.05271/) (2019)
- [AFlow: Automating Agentic Workflow Generation](https://mlsanity.com/paper/2410.10762/) (2024)
- [Cognitive Architectures for Language Agents](https://mlsanity.com/paper/2309.02427/) (2023)
- [Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models](https://mlsanity.com/paper/2510.04618/) (2025)
- [AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents](https://mlsanity.com/paper/2401.13178/) (2024)
- [SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine](https://mlsanity.com/paper/1704.05179/) (2017)
- [SkillOpt: Executive Strategy for Self-Evolving Agent Skills](https://mlsanity.com/paper/2605.23904/) (2026)
- [Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses](https://mlsanity.com/paper/2604.25850/) (2026)
- [SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulation](https://mlsanity.com/paper/2406.14991/) (2024)
- [Self-Harness: Harnesses That Improve Themselves](https://mlsanity.com/paper/2606.09498/) (2026)
- [SkillNet: Create, Evaluate, and Connect AI Skills](https://mlsanity.com/paper/2603.04448/) (2026)
- [OfficeQA Pro: An Enterprise Benchmark for End-to-End Grounded Reasoning](https://mlsanity.com/paper/2603.08655/) (2026)
- [SEW: Self-Evolving Agentic Workflows for Automated Code Generation](https://mlsanity.com/paper/2505.18646/) (2025)

## Built on by (most cited first, 9 of 9)

- [Ecdysis: Efficient and Effective Training of Runtime Harnesses for LLM Agents](https://mlsanity.com/paper/2609.11677/) (2026)
- [Bad Genius: Counterfactual-Guided Harness Evolution Beyond Task-Specific Shortcuts](https://mlsanity.com/paper/2609.18366/) (2026)
- [VERSE: Verified Self-Evolving Optimizer for Agent Harnesses](https://mlsanity.com/paper/2610.02616/) (2026)
- [Beyond Skill Evolution: Self-Evolving Context Management Policies for Long-Horizon Agent Harnesses](https://mlsanity.com/paper/2609.34649/) (2026)
- [VACE: Validation-Gated Alternating Co-Evolution of Agent Models and Harnesses](https://mlsanity.com/paper/2609.37105/) (2026)
- [Self-Evolving Harness on Multiple Tasks with the Agent as Its Own Optimizer](https://mlsanity.com/paper/2609.38372/) (2026)
- [GUI-HARVEST: Self-Improving GUI Agents through Evidence-Driven Harness Evolution](https://mlsanity.com/paper/2610.00948/) (2026)
- [From Existing Methods to New Research Ideas: Two Perspectives on Recursive Self-Improvement](https://mlsanity.com/paper/2610.14625/) (2026)
- [HarnessNet: Feedback-driven Hierarchical Joint Harness Evolution for Multi-agent Systems](https://mlsanity.com/paper/2610.15221/) (2026)

## Abstract (arXiv)

Self-evolving agents advance toward autonomy by optimizing their harness---prompts, skills, tools, and execution logic---based on environmental feedback. This paradigm, however, is hampered by three challenges: textitcredit assignment failure, where terminal success/failure feedback makes it ambiguous which step caused the error; textitshortcut learning, where agents memorize task-specific patterns rather than acquire generalizable capabilities; and textitcatastrophic forgetting, where unguarded updates degrade previously acquired competence. In this paper, we introduce HarnessEvolve, a self-evolving framework that learns from reference trajectories to achieve reliable agent self-evolution. HarnessEvolve decouples the execution agent from the evolutionary pipeline, assigning execution, evaluation, optimization, and gating to independent agent modules, enabling generalizable and stable harness improvements. Specifically, HarnessEvolve overcomes credit assignment failure by generating reference trajectories (execution paths produced when given the ground-truth answers) and aligning failed executions against them to extract error signals, which are clustered to reveal systematic failure patterns. To prevent shortcut learning and catastrophic forgetting, candidate harness updates must pass two gates: a quality gate that filters data leakage and prompt bloat, and a performance gate that accepts each update if it improves on the current batch without degrading recent batches, with epoch-end validation on a held-out set selecting the best-performing accepted agent snapshot. We conduct extensive experiments on several benchmarks spanning open-domain and enterprise scenarios, using different models and agent frameworks. Results demonstrate that HarnessEvolve consistently outperforms state-of-the-art baselines across all benchmarks and settings, confirming reliability across task domains.