# Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference

arXiv: https://arxiv.org/abs/2606.05922 · HTML: https://mlsanity.com/paper/2606.05922/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 43 references that are in the map.
- First posted on arXiv 4 Jun 2026.

## Builds on (most cited first, 15 of 15)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines](https://mlsanity.com/paper/2310.03714/) (2023)
- [Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces](https://mlsanity.com/paper/2601.11868/) (2026)
- [C-Pack: Packed Resources For General Chinese Embeddings](https://mlsanity.com/paper/2309.07597/) (2023)
- [Meta-Harness: End-to-End Optimization of Model Harnesses](https://mlsanity.com/paper/2603.28052/) (2026)
- [Sleep-time Compute: Beyond Inference Scaling at Test-time](https://mlsanity.com/paper/2504.13171/) (2025)
- [Gaia2: Benchmarking LLM Agents on Dynamic and Asynchronous Environments](https://mlsanity.com/paper/2602.11964/) (2026)
- [MemMA: Coordinating the Memory Cycle through Multi-Agent Reasoning and In-Situ Self-Evolution](https://mlsanity.com/paper/2603.18718/) (2026)
- [M^star: Every Task Deserves Its Own Memory Harness](https://mlsanity.com/paper/2604.11811/) (2026)
- [Towards Long-Horizon Interpretability: Efficient and Faithful Multi-Token Attribution for Reasoning LLMs](https://mlsanity.com/paper/2602.01914/) (2026)
- [Can LLMs Refuse Questions They Do Not Know? Measuring Knowledge-Aware Refusal in Factual Tasks](https://mlsanity.com/paper/2510.01782/) (2025)
- [MulVul: Retrieval-augmented Multi-Agent Code Vulnerability Detection via Cross-Model Prompt Evolution](https://mlsanity.com/paper/2601.18847/) (2026)
- [WebTrap: Stealthy Mid-Task Hijacking of Browser Agents During Navigation](https://mlsanity.com/paper/2605.08310/) (2026)

## Built on by (most cited first, 19 of 19)

- [JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolution](https://mlsanity.com/paper/2608.25593/) (2026)
- [AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Traces](https://mlsanity.com/paper/2608.23041/) (2026)
- [RRSI: Regularized Recursive Self-Improvement of Agent Harnesses](https://mlsanity.com/paper/2609.24972/) (2026)
- [WHALE: A Simple Recipe for Joint Harness-Weight Optimization](https://mlsanity.com/paper/2609.00196/) (2026)
- [ModularRSI: Modular and Generalizable Recursive Harness Self-Improvement](https://mlsanity.com/paper/2609.14857/) (2026)
- [Verify Smarter, Evolve Further: Efficient Harness Evolution through Behavior-Aware Verification](https://mlsanity.com/paper/2608.27311/) (2026)
- [NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness](https://mlsanity.com/paper/2609.08183/) (2026)
- [Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence](https://mlsanity.com/paper/2608.21156/) (2026)
- [Grow the Harness, Not the Context: From Strategy-Free Scaffolds to Reusable Specialist Agents](https://mlsanity.com/paper/2609.26760/) (2026)
- [EvoUndo: Recoverability-Constrained Self-Evolution for LLM Agent Harnesses](https://mlsanity.com/paper/2608.28363/) (2026)
- [Eureka: Task-Conditioned Meta-Agent Orchestration for Scientific Discovery](https://mlsanity.com/paper/2608.19047/) (2026)
- [CineForge: Self-Improving Agents for Long-Horizon Video Generation](https://mlsanity.com/paper/2608.29621/) (2026)
- [Why Git Is the Memory Solution for the Agentic Development Lifecycle](https://mlsanity.com/paper/2607.14390/) (2026)
- [RLHarness: Co-evolving Procedural Skills with Reinforcement Learning for Long-horizon Multimodal Reasoning](https://mlsanity.com/paper/2609.32326/) (2026)
- [Beyond Scripted Search: Sample-Efficient Reward Discovery via Agentic Black-box Optimization](https://mlsanity.com/paper/2609.32394/) (2026)
- [From Migration to Calibration: Aligning Agents with Deployment Requirements](https://mlsanity.com/paper/2609.35149/) (2026)
- [Turbo Harness: Instance-Adaptive Harness Optimization](https://mlsanity.com/paper/2609.40330/) (2026)
- [ActiveSaddler: Automated Curriculum Learning for Agent Harness Optimization](https://mlsanity.com/paper/2610.00906/) (2026)
- [ServeLearnBench: How Well Can Agents Self-Improve from Serving Experience?](https://mlsanity.com/paper/2610.07792/) (2026)

## Abstract (arXiv)

AI agents rely on a harness of skills, tools, and workflows to solve complex problems. Continually improving this harness is essential for adapting to new tasks. However, existing optimization methods typically require ground-truth validation sets, yet such labeled data is difficult to acquire in practical deployment settings. To address this problem, we introduce Retrospective Harness Optimization (RHO), a self-supervised method that optimizes the agent harness using only past trajectories. Specifically, RHO selects a diverse coreset of challenging tasks from past trajectories and re-solves them in parallel. The agent analyzes these rollouts using self-validation and self-consistency, then generates candidate harness updates and selects the most effective one by its own pairwise self-preference. We evaluate RHO across three diverse domains, spanning software engineering, technical work, and knowledge work. Notably, a single optimization round improves the pass rate on SWE-Bench Pro from 59% to 78% without any external grading. Furthermore, our analysis demonstrates that RHO effectively targets prior failure modes. As a result, the optimized harness alters the agent's behavior patterns and sustains higher accuracy during long-horizon sessions.