mlsanityData to 14 Oct 2026

EvoTrainer: Co-Evolving LLM Policies and Training Harnesses for Autonomous Agentic Reinforcement Learning

arXiv 2606.03108 ↗First posted 2 Jun 2026cs.AIcited by 14

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Autonomous LLM training is often framed as recipe search, which leaves the training harness largely static. This limitation sharpens in agentic RL, where shifting bottlenecks and scalar rewards mask diverse failure modes. We introduce EvoTrainer, an autonomous training framework that co-evolves LLM policies and training-side harnesses through empirical feedback: it diagnoses rollout-level evidence, revises diagnostics, backtests interventions, and accumulates reusable skills. Evaluated on mathematical reasoning, competitive-programming code generation, and repository-level software engineering, EvoTrainer matches or exceeds the human-engineered RL references under the same data, codebase, and evaluation protocol, with the largest gain on long-horizon agentic SWE. Trajectory analyses show that retained strategies diverge across domains, evolving diagnostics prevent invalid high-scoring branches from being promoted, and reusable skills shape later search. Autonomous LLM RL should move beyond recipe search toward joint evolution of policies and the training harnesses that interpret them.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 27 of its 34 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

27 of 34 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026Demystifying Reinforcement Learning for Long-Horizon Tool-Using Agents: A Comprehensive Recipe23 Mar
  2. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning24 Apr
  3. 2025Reinforcement Learning for Long-Horizon Interactive LLM Agents3 Feb

Other references · 24

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  3. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  4. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
  5. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
  6. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 867
  7. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  8. 2017Deep Reinforcement Learning that Matterscited by 754
  9. 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Searchcited by 422
  10. 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 356
  11. 2026Meta-Harness: End-to-End Optimization of Model Harnessescited by 309
  12. 2025R-Zero: Self-Evolving Reasoning LLM from Zero Datacited by 228
12 more
  1. 2023TACO: Topics in Algorithmic COde generation datasetcited by 160
  2. 2026Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnessescited by 142
  3. 2025Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Modelscited by 109
  4. 2018How Many Random Seeds? Statistical Power Analysis in Deep Reinforcement Learning Experimentscited by 60
  5. 2025RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environmentscited by 50
  6. 2025GHPO: Adaptive Guidance for Stable and Efficient LLM Reinforcement Learningcited by 34
  7. 2026RAGEN-2: Reasoning Collapse in Agentic RLcited by 20
  8. 2025DRA-GRPO: Your GRPO Needs to Know Diverse Reasoning Paths for Mathematical Reasoningcited by 18
  9. 2026On the Emergence of Implicit Curriculum in RLVR Learning Dynamicscited by 10
  10. 2026Auto Research with Specialist Agents Develops Effective and Non-Trivial Training Recipescited by 10
  11. 2026Beyond Quantity: Trajectory Diversity Scaling for Code Agentscited by 5
  12. 2026GEAR: Genetic AutoResearch for Agentic Code Evolutioncited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 14.

14 papers.

Sort:

By area:

  1. 2026SafeEvolve: Harness-Policy Co-Evolution from Agent Experience for Safety Alignmentcited by 6
  2. 2026You Don't Need To Stay in The Loop: An Agentic Robotics Loop for Robot-Policy Improvementcited by 2
  3. 2026What is Missing from AI Post-Training AI: An Empirical Analysiscited by 2
  4. 2026MetaRSI / RSI2: A Meta-Recursive Self-Improving System for Recursive Self-Improving Systems Themselvescited by 2
  5. 2026EvoRS: On-Policy Self-Evolution of Reward Systems for Open-Ended Reinforcement Learningcited by 1
  6. 2026VERSE: Verified Self-Evolving Optimizer for Agent Harnessescited by 1
  7. 2026CoTrace: Data Recipes for Training Terminal Agents with Harness-Model Co-Evolutioncited by 1
  8. 2026Dual-Loop Self-Evolution via Verifiable Emotion Feedback for Multi-Turn Empathetic Dialoguecited by 0
  9. 2026Knowing When Not to Reuse: Conditional Experience Transfer in Autonomous LLM Post-Trainingcited by 0
  10. 2026Physis-Lang: Self-Evolving Language as a Physical Representation for Video World Modelcited by 0
  11. 2026From Existing Methods to New Research Ideas: Two Perspectives on Recursive Self-Improvementcited by 0
  12. 2026EAU: Environment-Agent-User Co-evolution for Training User-Centric Agentscited by 0
2 more
  1. 2026Cowork-Gym: Synthesizing Interactive Work Environments at Scalecited by 0
  2. 2026Coupled Harness--Model Evolution for Scientific Equation Discoverycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents
  2. 2026Teaching LLMs to Self-Evolve: Cultivating Core Meta-Skills with Reinforcement Learning
  3. 2026Code Evolution for Control: Synthesizing Policies via LLM-Driven Evolutionary Search
  4. 2026A Control System, a Dataset, and a Recipe for Making Frozen LLM Agents Learn a Domain
  5. 2026SynCo: Data Synthesis Co-Training for Self-Evolving LLMs via Multi-Agent Reinforcement Learning
  6. 2026HarnessForge: Joint Harness and Policy Evolution for Adaptive Agent Systems
  7. 2026From Memory to Skills: Evidence-Grounded Co-Evolution Governance for Long-Horizon LLM Agents
  8. 2026SkillForge: Co-Evolving Skills and Agents via Dynamic Skill Lifecycles