mlsanityData to 14 Oct 2026

Verifier-Harness Co-Evolution for Training-Free Self-Improvement on Open-Ended Tasks

arXiv 2610.17609 ↗First posted 13 Oct 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Training-free self-improvement lets an agent use feedback to revise the response it is about to return and to update the harness state that shapes how it handles later tasks. On open-ended tasks that feedback rarely comes from reference answers or executable tests, so the agent has to rely on a model-based verifier. When the verifier is wrong, the current response can get worse, and misleading experience can persist across the task stream. A verifier that never changes also grows stale as the agent's responses improve and task demands shift. We introduce Unified Co-Evolution (UCoE), a training-free method that evolves an executable verifier program together with the agent's harness state while all model parameters stay fixed. UCoE synthesizes candidate verifier programs and tests them on response contrasts, pairs of responses that differ in one declared property and have a known preferred side. It selects the candidate whose discrimination on its weakest contrast family is highest, a quantity we call driving capacity. The selected verifier diagnoses one weakness in each draft, guides a single repair, decides which response is served, and determines whether the accepted repair may enter the harness state. As tasks complete, their diagnostics seed new candidate verifiers, which must beat the current verifier on the same contrasts before adoption. Across six open-ended benchmarks, UCoE averages 59.00% preference against matched direct generation with Qwen3-8B and 57.19% with Qwen3.8-27B, margins of 9.00 and 7.19 percentage points. On three verifiable benchmarks, the same method raises the mean from 36.06% for direct generation to 38.48%. Code is available at https://github.com/tianyi0216/Unified-Co-Evolution.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 42 of its 54 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

42 of 54 references.

References · 42

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  4. 2023Let's Verify Step by Stepcited by 3,789
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  6. 2018Hierarchical Neural Story Generationcited by 1,193
  7. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,063
  8. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 876
  9. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 571
  10. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 546
  11. 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 500
  12. 2025Agentic Context Engineering: Evolving Contexts for Self-Improving Language Modelscited by 398
30 more
  1. 2025Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domainscited by 366
  2. 2024WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wildcited by 218
  3. 2025Generalizing Verifiable Instruction Followingcited by 194
  4. 2025MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMscited by 180
  5. 2025Dynamic Cheatsheet: Test-Time Learning with Adaptive Memorycited by 163
  6. 2025DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Researchcited by 114
  7. 2025Reinforcement Learning with Rubric Anchorscited by 113
  8. 2025WritingBench: A Comprehensive Benchmark for Generative Writingcited by 106
  9. 2025Scaling Test-time Compute for LLM Agentscited by 63
  10. 2025Multi-Agent Verification: Scaling Test-Time Compute with Multiple Verifierscited by 62
  11. 2026Rethinking Rubric Generation for Improving LLM Judge and Reward Modeling for Open-ended Taskscited by 62
  12. 2026Alternating Reinforcement Learning for Rubric-Based Reward Modeling in Non-Verifiable LLM Post-Trainingcited by 54
  13. 2021Perturbation CheckLists for Evaluating NLG Evaluation Metricscited by 50
  14. 2025Writing-Zero: Bridge the Gap Between Non-verifiable Tasks and Verifiable Rewardscited by 50
  15. 2026Recursive Harness Self-Improvementcited by 32
  16. 2026Inference-Time Scaling of Verification: Self-Evolving Deep Research Agents via Test-Time Rubric-Guided Verificationcited by 22
  17. 2026The Red Queen G"odel Machine: Co-Evolving Agents and Their Evaluatorscited by 19
  18. 2025RefineBench: Evaluating Refinement Capability of Language Models via Checklistscited by 15
  19. 2025When Does Verification Pay Off? A Closer Look at LLMs as Solution Verifierscited by 15
  20. 2026CoVerRL: Breaking the Consensus Trap in Label-Free Reasoning via Generator-Verifier Co-Evolutioncited by 12
  21. 2026EvoRubric: Self-Evolving Rubric-Driven RL for Open-Ended Generationcited by 11
  22. 2026SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflowscited by 11
  23. 2026SCOPE: Self-Play via Co-Evolving Policies for Open-Ended Taskscited by 7
  24. 2026GenRubric: Self-Evolving Rubric Generation for Scalable LLM Evaluationcited by 5
  25. 2026FUSE: Ensembling Verifiers with Zero Labeled Datacited by 4
  26. 2026EVOCHAMBER: Test-Time Co-evolution of Multi-Agent System at Individual, Team, and Population Scalescited by 4
  27. 2026EvoPool: Evolutionary Programmatic Annotation for Label-Efficient Specialized Supervisioncited by 1
  28. 2026From RLVR to RLSVR: Task Transformation Induces Self-Verifiable Rewards for Open-Ended LLM Self-Improvementcited by 1
  29. 2026EvoRS: On-Policy Self-Evolution of Reward Systems for Open-Ended Reinforcement Learningcited by 1
  30. 2026EvalResearchBench: Can AI Agents Design Their Own Evaluations?cited by 1

Built on by

Not cited yet.