mlsanityData to 7 Oct 2026

CoEvolve: Training LLM Agents via Agent-Data Mutual Evolution

arXiv 2604.15840 ↗First posted 17 Apr 2026cs.CLcited by 14

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Reinforcement learning for LLM agents is typically conducted on a static data distribution, which fails to adapt to the agent's evolving behavior and leads to poor coverage of complex environment interactions. To address these challenges, we propose CoEvolve, an agent-data mutual evolution framework that enables LLM agents to improve through closed-loop, interaction-driven training. Specifically, CoEvolve extracts feedback signals such as forgetting and uncertainty from rollout trajectories to identify failure-prone interaction patterns, and utilizes them to guide LLM-based task synthesis. The synthesized tasks are validated through environment interaction and utilized to update the data distribution, enabling joint adaptation of the agent and its data. Extensive experiments on AppWorld and BFCL across Qwen2.5-7B, Qwen3-4B, and Qwen3-30B-A3B demonstrate consistent and significant improvements over strong base models, yielding absolute gains of 19.43%, 15.58%, and 18.14%, respectively.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 28 of its 47 references that are in the map. First posted on arXiv 17 Apr 2026.

Builds on

28 of 47 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Demystifying Reinforcement Learning in Agentic Reasoning13 Oct
  2. 2025STeCa: Step-level Trajectory Calibration for LLM Agent Learning20 Feb

Other references · 26

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  4. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  5. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  6. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  7. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
  8. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  9. 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 359
  10. 2025GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoningcited by 147
  11. 2024AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorialscited by 123
  12. 2025Agent-R: Training Language Model Agents to Reflect via Iterative Self-Trainingcited by 74
14 more
  1. 2025DeepAgent: A General Reasoning Agent with Scalable Toolsetscited by 69
  2. 2025Explorer: Scaling Exploration-driven Web Trajectory Synthesis for Multimodal Web Agentscited by 68
  3. 2024AgentBank: Towards Generalized LLM Agents via Fine-Tuning on 50000+ Interaction Trajectoriescited by 56
  4. 2025DeepAnalyze: Agentic Large Language Models for Autonomous Data Sciencecited by 49
  5. 2025A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applicationscited by 21
  6. 2025Scaling Synthetic Task Generation for Agents via Explorationcited by 19
  7. 2023A Comprehensive Survey on Rare Event Predictioncited by 18
  8. 2025Stepwise Guided Policy Optimization: Coloring your Incorrect Reasoning in GRPOcited by 18
  9. 2025LIMI: Less is More for Agencycited by 17
  10. 2025Training LLM-Based Agents with Synthetic Self-Reflected Trajectories and Partial Maskingcited by 16
  11. 2025ComPO: Preference Alignment via Comparison Oraclescited by 11
  12. 2025LAM SIMULATOR: Advancing Data Generation for Large Action Model Training via Online Exploration and Trajectory Feedbackcited by 6
  13. 2025An Empirical Study on How Video-LLMs Answer Video Questionscited by 4
  14. 2025Where and What Matters: Sensitivity-Aware Task Vectors for Many-Shot Multimodal In-Context Learningcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 14.

14 papers.

Sort:

By area:

  1. 2026Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?cited by 11
  2. 2026SEAL: Synergistic Co-Evolution of Agents and Learning Environmentscited by 9
  3. 2026LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Taskscited by 9
  4. 2026Towards On-Policy Data Evolution for Visual-Native Multimodal Deep Search Agentscited by 6
  5. 2026Keep Policy Gradient in Charge: Sibling-Guided Credit Distillation for Long-Horizon Tool-Use Agentscited by 6
  6. 2026Simulate to Generalize: Scaling Stateful Supervision for API-calling Agents using LLM World Modelscited by 2
  7. 2026Environment Evolution for Terminal Agentscited by 2
  8. 2026RISE: Reliable Improvement in Self-Evolving Vision-Language Modelscited by 1
  9. 2026Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Designcited by 1
  10. 2026D2Evo: Dual Difficulty-Aware Self-Evolution for Data-Efficient Reinforcement Learningcited by 0
  11. 2026Dual-Loop Self-Evolution via Verifiable Emotion Feedback for Multi-Turn Empathetic Dialoguecited by 0
  12. 2026EvoRS: On-Policy Self-Evolution of Reward Systems for Open-Ended Reinforcement Learningcited by 0
2 more
  1. 2026AutoDataBench: Can Agents Write the Data That Feeds the Self-Improvement Loop?cited by 0
  2. 2026ActiveSaddler: Automated Curriculum Learning for Agent Harness Optimizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Learn as Individuals, Evolve as a Team: Multi-agent LLMs Adaptation in Embodied Environments
  2. 2025Multi-Agent Evolve: LLM Self-Improve through Co-evolution
  3. 2024Coevolving with the Other You: Fine-Tuning LLM with Sequential Cooperative Multi-Agent Reinforcement Learning
  4. 2026EvoTrainer: Co-Evolving LLM Policies and Training Harnesses for Autonomous Agentic Reinforcement Learning
  5. 2026Cooperative Coevolution for Resource-Constrained Agentic LLM Post-Training
  6. 2026LLM-ALSO: LLM-Driven Adaptive Learning-Signal Optimization for Multi-Agent Reinforcement Learning
  7. 2026Teaching LLMs to Self-Evolve: Cultivating Core Meta-Skills with Reinforcement Learning
  8. 2026From Memory to Skills: Evidence-Grounded Co-Evolution Governance for Long-Horizon LLM Agents