mlsanityData to 7 Oct 2026

VeRO: A Harness for Agents to Optimize Agents

arXiv 2602.22480 ↗First posted 25 Feb 2026cs.AI cs.CL cs.LGcited by 12

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

An important emerging application of coding agents is agent harness optimization: the iterative improvement of a target agent by editing and evaluating its code. Despite its relevance, the community lacks a systematic understanding of coding agent performance on this task. Harness optimization differs from conventional software engineering: agent harnesses interleave deterministic code with stochastic LLM completions, requiring structured capture of both intermediate execution traces and downstream outcomes. To address these challenges, we introduce (1) VeRO (Versioning, Rewards, and Observations), an outer harness that provides versioned snapshots, budget-controlled evaluation, and structured execution traces of target harnesses, and (2) VeRO-Bench, a benchmark suite of target agents and tasks with reference evaluation procedures. Using VeRO, we conduct an empirical study comparing optimizers across tasks and analyzing which modifications reliably improve target agent harnesses. We release VeRO to support research on agent optimization as a core capability for coding agents. Code is available at https://github.com/scaleapi/vero.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 24 of its 47 references that are in the map. First posted on arXiv 25 Feb 2026.

Builds on

24 of 47 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025AlphaEvolve: A coding agent for scientific and algorithmic discovery16 Jun
  2. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agents23 Jul
  3. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering6 May

Other references · 21

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2021Program Synthesis with Large Language Modelscited by 3,509
  5. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  6. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  7. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  8. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  9. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  10. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  11. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  12. 2023Large Language Models as Optimizerscited by 830
9 more
  1. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  2. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 515
  3. 2024Automated Design of Agentic Systemscited by 360
  4. 2024Measuring short-form factuality in large language modelscited by 345
  5. 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 294
  6. 2023Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generationcited by 128
  7. 2025LLM-AutoDiff: Auto-Differentiate Any LLM Workflowcited by 19
  8. 2025The FACTS Leaderboard: A Comprehensive Benchmark for Large Language Model Factualitycited by 14
  9. 2025The Adoption and Usage of AI Agents: Early Evidence from Perplexitycited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 12.

12 papers.

  1. 2026HarnessBank: Semantic Gene-Bank Search with Gated Verification for Agent-Harness Self-Evolutioncited by 11
  2. 2026Evo-Bench: Can Language Models Improve Agent Harness?cited by 8
  3. 2026HarnessDev: Can LLMs Create and Evolve Their Own Agent Harness?cited by 7
  4. 2026HarnessOpt-Bench: Evaluating LLMs at Harness Optimizationcited by 5
  5. 2026Towards Direct Evaluation of Harness Optimizers via Priority Rankingcited by 2
  6. 2026EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agentscited by 2
  7. 2026Grow the Harness, Not the Context: From Strategy-Free Scaffolds to Reusable Specialist Agentscited by 2
  8. 2026Insights Generator: Systematic Corpus-Level Trace Diagnostics for LLM Agentscited by 1
  9. 2026Beyond Final Scores: A Systematic Evaluation of Agents for Long-Horizon AI Research and Developmentcited by 1
  10. 2026Studying Without a Syllabus: Task-Agnostic Environment Preprocessingcited by 0
  11. 2026VideoResearcher: Self-Improving Tool Design for Long-Video Understandingcited by 0
  12. 2026OSFoundry: Building and Evolving Operating Systems with Specification-Guided Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Don't Blame the Large Language Model: How Agent Harness Evolution Shapes Coding Agent Quality
  2. 2026Code as Agent Harness
  3. 2026PERFOPT-Bench: Evaluating Coding Agents on Software Performance Optimization
  4. 2026An Empirical Study of Harness Design for Coding Agents
  5. 2026EvoCode-Bench: Evaluating Coding Agents in Multi-Turn Iterative Interactions
  6. 2026EffiSkill: Agent Skill Based Automated Code Efficiency Optimization
  7. 2026Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems
  8. 2026Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflows