# Task-CoEvolve: Efficient Harness Optimization via Adaptive Validation Task Selection

arXiv: https://arxiv.org/abs/2608.20169 · HTML: https://mlsanity.com/paper/2608.20169/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 38 references that are in the map.
- First posted on arXiv 20 Aug 2026.

## Builds on (most cited first, 25 of 25)

- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence](https://mlsanity.com/paper/2606.19348/) (2026)
- [STaR: Bootstrapping Reasoning With Reasoning](https://mlsanity.com/paper/2203.14465/) (2022)
- [Self-Rewarding Language Models](https://mlsanity.com/paper/2401.10020/) (2024)
- [Large Language Models Can Self-Improve](https://mlsanity.com/paper/2210.11610/) (2022)
- [Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces](https://mlsanity.com/paper/2601.11868/) (2026)
- [Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models](https://mlsanity.com/paper/2510.04618/) (2025)
- [tinyBenchmarks: evaluating LLMs with fewer examples](https://mlsanity.com/paper/2402.14992/) (2024)
- [Meta-Harness: End-to-End Optimization of Model Harnesses](https://mlsanity.com/paper/2603.28052/) (2026)
- [Curriculum Learning: A Survey](https://mlsanity.com/paper/2101.10382/) (2021)
- [ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution](https://mlsanity.com/paper/2509.19349/) (2025)
- [Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses](https://mlsanity.com/paper/2604.25850/) (2026)
- [LawBench: Benchmarking Legal Knowledge of Large Language Models](https://mlsanity.com/paper/2309.16289/) (2023)
- [Self-Adapting Language Models](https://mlsanity.com/paper/2506.10943/) (2025)
- [Active Testing: Sample-Efficient Model Evaluation](https://mlsanity.com/paper/2103.05331/) (2021)
- [Meta Context Engineering via Agentic Skill Evolution](https://mlsanity.com/paper/2601.21557/) (2026)
- [Rethinking the Evaluation of Harness Evolution for Agents](https://mlsanity.com/paper/2607.12227/) (2026)
- [AcTracer: Active Testing of Large Language Model via Multi-Stage Sampling](https://mlsanity.com/paper/2408.03573/) (2024)
- [HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnesses](https://mlsanity.com/paper/2608.01918/) (2026)
- [TTHE: Test-Time Harness Evolution](https://mlsanity.com/paper/2607.08124/) (2026)
- [SWE-Bench Mobile: Can Large Language Model Agents Develop Industry-Level Mobile Applications?](https://mlsanity.com/paper/2602.09540/) (2026)
- [Harnessing Agentic Evolution](https://mlsanity.com/paper/2605.13821/) (2026)
- [DemoEvolve: Overcoming Sparse Feedback in Agentic Harness Evolution with Demonstrations](https://mlsanity.com/paper/2605.24539/) (2026)
- [DREvo: Distilling Recalibrated Historical Experience for Harness Self-Evolution](https://mlsanity.com/paper/2607.26722/) (2026)
- [TurboEvolve: Towards Fast and Robust LLM-Driven Program Evolution](https://mlsanity.com/paper/2604.18607/) (2026)

## Built on by (most cited first, 5 of 5)

- [More Programs or More Rolls? Separating Coverage from Specialization in LLM Harnesses](https://mlsanity.com/paper/2609.35873/) (2026)
- [Scaling Trajectories for Complex Tasks through Recursive Self-Rewrite](https://mlsanity.com/paper/2610.02826/) (2026)
- [AutoRef: Harness Optimization for Agentic Multi-Reference Image Generation](https://mlsanity.com/paper/2609.35530/) (2026)
- [ActiveSaddler: Automated Curriculum Learning for Agent Harness Optimization](https://mlsanity.com/paper/2610.00906/) (2026)
- [HERA: Harness-Environment Co-Evolution for Reliable Agentic Abstention](https://mlsanity.com/paper/2610.06563/) (2026)

## Abstract (arXiv)

We present a novel approach to efficient LLM harness optimization through adaptive validation task selection. Harness optimization iteratively rewrites the harness code based on validation performance, enabling substantial performance gains without updating the underlying model weights. Existing approaches, however, evaluate a fixed validation set in full at every iteration, incurring substantial evaluation costs even on tasks that become less discriminative as the harness evolves. We propose Task-CoEvolve, which co-evolves the validation tasks with the harness by addressing two challenges: selecting informative tasks and estimating full-set performance from partial evaluations. Task-CoEvolve builds on the observation that tasks on which candidate harnesses disagree are more informative for distinguishing among them than tasks that are consistently solved or failed. It uses variance-weighted sampling based on past outcomes to focus evaluation on tasks near the capability frontier, with the sampling distribution adapting as the harness evolves. It then estimates full-set scores from the sampled tasks by accounting for their sampling probabilities, enabling consistent comparisons across iterations despite evaluating different subsets. Experiments on online text classification and Terminal-Bench 2.1 show that Task-CoEvolve consistently outperforms subset-based baselines and matches the final performance of full-set search while reducing the number of evaluations during optimization by 80%. Code will be released at https://github.com/Agent4Science-UTokyo/Task-CoEvolve.