mlsanityData to 7 Oct 2026

Multi-Agent Evolve: LLM Self-Improve through Co-evolution

arXiv 2510.23595 ↗First posted 27 Oct 2025cs.AIcited by 58

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 2 more
Abstract (arXiv)

Reinforcement Learning (RL) has demonstrated significant potential in enhancing the reasoning capabilities of large language models (LLMs). However, the success of RL for LLMs heavily relies on human-curated datasets and verifiable rewards, which limit their scalability and generality. Recent Self-Play RL methods, inspired by the success of the paradigm in games and Go, aim to enhance LLM reasoning capabilities without human-annotated data. However, their methods primarily depend on a grounded environment for feedback (e.g., a Python interpreter or a game engine); extending them to general domains remains challenging. To address these challenges, we propose Multi-Agent Evolve (MAE), a framework that enables LLMs to self-evolve in solving diverse tasks, including mathematics, reasoning, and general knowledge Q&A. The core design of MAE is based on a triplet of interacting agents (Proposer, Solver, Judge) that are instantiated from a single LLM, and applies reinforcement learning to optimize their behaviors. The Proposer generates questions, the Solver attempts solutions, and the Judge evaluates both while co-evolving. Experiments on Qwen2.5-3B-Instruct demonstrate that MAE achieves an average improvement of 4.54% on multiple benchmarks. These results highlight MAE as a scalable, data-efficient method for enhancing the general reasoning abilities of LLMs with minimal reliance on human-curated supervision.

Key facts. Cited by 58 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 41 of its 75 references that are in the map. First posted on arXiv 27 Oct 2025.

Builds on

41 of 75 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025R-Zero: Self-Evolving Reasoning LLM from Zero Data7 Aug
  2. 2025SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learning30 Jun
  3. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate30 May

Other references · 38

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  8. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  9. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  10. 2021Program Synthesis with Large Language Modelscited by 3,509
  11. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  12. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
26 more
  1. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  2. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  3. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  4. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  5. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  6. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  7. 2024A Survey on LLM-as-a-Judgecited by 1,384
  8. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  9. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  10. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  11. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
  12. 2024Self-Rewarding Language Modelscited by 654
  13. 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
  14. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
  15. 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agentscited by 329
  16. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 323
  17. 2022Large Language Models are Better Reasoners with Self-Verificationcited by 304
  18. 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attackscited by 135
  19. 2023Language Model Self-improvement by Reinforcement Learning Contemplationcited by 75
  20. 2025Self-Challenging Language Model Agentscited by 56
  21. 2025SeRL: Self-Play Reinforcement Learning for Large Language Models with Limited Datacited by 50
  22. 2025Learning to Solve and Verify: A Self-Play Framework for Code and Test Generationcited by 34
  23. 2025Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Modelscited by 33
  24. 2022CAPE: Corrective Actions from Precondition Errors using Large Language Modelscited by 30
  25. 2024Simulating Human-like Daily Activities with Desire-driven Autonomycited by 30
  26. 2024Scalable Reinforcement Post-Training Beyond Static Human Prompts: Evolving Alignment via Asymmetric Self-Playcited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 53.

58 papers.

Sort:

By area:

  1. 2026MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agentscited by 129
  2. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  3. 2025VisPlay: Self-Evolving Vision-Language Models from Imagescited by 42
  4. 2026A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agentscited by 40
  5. 2025EvoLMM: Self-Evolving Large Multimodal Models with Continuous Rewardscited by 24
  6. 2026UniCorn: Towards Self-Improving Unified Multimodal Models through Self-Generated Supervisioncited by 23
  7. 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skillscited by 19
  8. 2026Self-Improvements in Modern Agentic Systems: A Surveycited by 17
  9. 2026Self-Play Only Evolves When Self-Synthetic Pipeline Ensures Learnable Information Gaincited by 16
  10. 2026The Red Queen G"odel Machine: Co-Evolving Agents and Their Evaluatorscited by 16
  11. 2026Does Socialization Emerge in AI Agent Society? A Case Study of Moltbookcited by 13
  12. 2026EvolveRouter: Co-Evolving Routing and Prompt for Multi-Agent Question Answeringcited by 13
38 more
  1. 2026Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?cited by 11
  2. 2026One Model, All Roles: Multi-Turn, Multi-Agent Self-Play Reinforcement Learning for Conversational Social Intelligencecited by 7
  3. 2026Self-Play Meets Skill Evolution: Self-Evolving Search Agents that Pose, Solve, and Remembercited by 7
  4. 2026The Devil Behind Moltbook: Anthropic Safety is Always Vanishing in Self-Evolving AI Societiescited by 6
  5. 2026PrivAct: Internalizing Contextual Privacy Preservation via Multi-Agent Preference Trainingcited by 6
  6. 2026SAGE: Multi-Agent Self-Evolution for LLM Reasoningcited by 6
  7. 2026Code-A1: Adversarial Evolving of Code LLM and Test LLM via Reinforcement Learningcited by 6
  8. 2026Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skillscited by 6
  9. 2026Self-Improvement of Large Language Models: A Technical Overview and Future Outlookcited by 5
  10. 2026Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systemscited by 5
  11. 2026Reinforcement Learning for LLM-based Multi-Agent Systems through Orchestration Tracescited by 4
  12. 2026ElasticMem: Latent Memory as a Learnable Resource for LLM Agentscited by 4
  13. 2026Role-Agent: Bootstrapping LLM Agents via Dual-Role Evolutioncited by 4
  14. 2025PersonalizedRouter: Personalized LLM Routing via Graph-based User Preference Modelingcited by 3
  15. 2026Stabilizing Unsupervised Self-Evolution of MLLMs via Continuous Softened Retracing reSamplingcited by 3
  16. 2026CascadeDebate: Multi-Agent Deliberation for Cost-Aware LLM Cascadescited by 3
  17. 2026EVOCHAMBER: Test-Time Co-evolution of Multi-Agent System at Individual, Team, and Population Scalescited by 3
  18. 2026Economy of Minds: Emerging Multi-Agent Intelligence with Economic Interactionscited by 3
  19. 2026Towards AGI A Pragmatic Approach Towards Self Evolving Agentcited by 2
  20. 2026Self-evolving Embodied AIcited by 2
  21. 2026MAGE: Multi-Agent Self-Evolution with Co-Evolutionary Knowledge Graphscited by 2
  22. 2026EvoRepair: Enhancing Vulnerability Repair Agents Through Experience-Based Self-Evolutioncited by 2
  23. 2026Matryoshka Agent: Unfolding Sub-Agents for Long-Horizon Machine Learning Engineeringcited by 2
  24. 2026Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligencecited by 2
  25. 2026Vocabulary Dropout for Curriculum Diversity in LLM Co-Evolutioncited by 1
  26. 2026Superminds Test: Actively Evaluating Collective Intelligence of Agent Society via Probing Agentscited by 1
  27. 2026Agentic AI for Remote Sensing: Technical Challenges and Research Directionscited by 1
  28. 2026MetaAgent-X : Breaking the Ceiling of Automatic Multi-Agent Systems via End-to-End Reinforcement Learningcited by 1
  29. 2026Multi-Agent Coordination Adaptation via Structure-Guided Orchestrationcited by 1
  30. 2026Self-Verified Distillation: Your Language Model Is Secretly Its Own Synthetic Data Pipelinecited by 1
  31. 2026A Primer in Post-Training Reasoning Data: What We Know About How It Workscited by 1
  32. 2026Escaping the Self-Confirmation Trap: An Execute-Distill-Verify Paradigm for Agentic Experience Learningcited by 1
  33. 2026Self-Modifying Lean Proof Agents with Verifier-Grounded Benchmark Coevolutioncited by 1
  34. 2026HypoEvolve: Genetic Algorithms Enable Multi-Agent LLMs to Discover Scientific Hypothesescited by 1
  35. 2025One Request, Multiple Experts: LLM Orchestrates Domain Specific Models via Adaptive Task Routingcited by 0
  36. 2026A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutionscited by 0
  37. 2026JigsawRL: Assembling RL Pipelines for Efficient LLM Post-Trainingcited by 0
  38. 2026AIPO: Learning to Reason from Active Interactioncited by 0

50 most cited of 58 All 58 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Learn as Individuals, Evolve as a Team: Multi-agent LLMs Adaptation in Embodied Environments
  2. 2025CoMAS: Co-Evolving Multi-Agent Systems via Interaction Rewards
  3. 2025Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning
  4. 2024A Survey on Self-Evolution of Large Language Models
  5. 2025MARSHAL: Incentivizing Multi-Agent Reasoning via Self-Play with Strategic LLMs
  6. 2025EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecycle
  7. 2026PopuLoRA: Co-Evolving LLM Populations for Reasoning Self-Play
  8. 2025The Path of Self-Evolving Large Language Models: Achieving Data-Efficient Learning via Intrinsic Feedback