mlsanityData to 7 Oct 2026

Stronger-MAS: Multi-Agent Reinforcement Learning for Collaborative LLMs

arXiv 2510.11062 ↗First posted 13 Oct 2025cs.LG cs.MAcited by 25

On the map: LLM reasoning → Reinforcement learning for reasoning

also in 1 more
Abstract (arXiv)

Multi-agent systems (MAS) and reinforcement learning (RL) are widely used to enhance the agentic capabilities of large language models (LLMs). MAS improves task performance through role-based orchestration, while RL uses environmental rewards to learn stronger policies, such as GRPO-style optimization. However, applying on-policy RL to MAS remains underexplored and presents unique challenges. Algorithmically, standard GRPO grouping assumptions break down because prompts vary by role and by turn. System-wise, the training stack must support MAS-workflow rollouts and on-policy updates for both single-policy and multi-policy models. We propose AT-GRPO, which includes (i) an agent- and turn-wise grouped RL algorithm tailored to MAS and (ii) a training system that supports both single- and multi-policy regimes. Across game, planning, coding, and math tasks, AT-GRPO delivers substantial gains. On long-horizon planning, it increases accuracy from a 14.0 to 47.0 percent single-agent RL baseline to 96.0 to 99.5 percent. It also improves reasoning performance, with average gains of 3.87 to 7.62 percent on coding tasks and 9.0 to 17.93 percent on math. Code and environments are available at: https://github.com/pettingllms-ai/PettingLLMs.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 35 of its 45 references that are in the map. First posted on arXiv 13 Oct 2025.

Builds on

35 of 45 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Group-in-Group Policy Optimization for LLM Agent Training16 May

Other references · 34

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  6. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  7. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  8. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  9. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  10. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  11. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  12. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
22 more
  1. 2021Measuring Coding Challenge Competence With APPScited by 879
  2. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 752
  3. 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
  4. 2024Mixture-of-Agents Enhances Large Language Model Capabilitiescited by 420
  5. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 357
  6. 2025ToolRL: Reward is All Tool Learning Needscited by 342
  7. 2024MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolutioncited by 132
  8. 2025SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learningcited by 89
  9. 2024Internet of Agents: Weaving a Web of Heterogeneous Agents for Collaborative Intelligencecited by 80
  10. 2025MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learningcited by 80
  11. 2025SETS: Leveraging Self-Verification and Self-Correction for Improved Test-Time Scalingcited by 54
  12. 2025lmgame-Bench: How Good are LLMs at Playing Games?cited by 54
  13. 2025MARFT: Multi-Agent Reinforcement Fine-Tuningcited by 51
  14. 2025Co-Evolving LLM Coder and Unit Tester via Reinforcement Learningcited by 51
  15. 2025OrcaLoca: An LLM Agent Framework for Software Issue Localizationcited by 43
  16. 2024DrugAgent: Automating AI-aided Drug Discovery Programming through LLM Multi-Agent Collaborationcited by 40
  17. 2025X-MAS: Towards Building Multi-Agent Systems with Heterogeneous LLMscited by 33
  18. 2024Coevolving with the Other You: Fine-Tuning LLM with Sequential Cooperative Multi-Agent Reinforcement Learningcited by 32
  19. 2025CodeSteer: Symbolic-Augmented Language Models via Code/Text Guidancecited by 11
  20. 2020Multi-Agent Reinforcement Learning for Problems with Combined Individual and Team Rewardcited by 8
  21. 2025End-to-End Optimization of LLM-Driven Multi-Agent Search Systems via Heterogeneous-Group-Based Reinforcement Learningcited by 7
  22. 2020Reward Design in Cooperative Multi-agent Reinforcement Learning for Packet Routingcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 25.

25 papers.

Sort:

By area:

  1. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  2. 2026Exact Is Easier: Credit Assignment for Cooperative LLM Agentscited by 9
  3. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  4. 2026Learning Decentralized LLM Collaboration with Multi-Agent Actor Criticcited by 7
  5. 2026SAGE: Multi-Agent Self-Evolution for LLM Reasoningcited by 6
  6. 2026Multimodal Model for Computational Pathology:Representation Learning and Image Compressioncited by 6
  7. 2026UnityMAS-O: A General RL Optimization Framework for LLM-Based Multi-Agent Systemscited by 6
  8. 2026Opt-Verifier: Unleashing the Power of LLMs for Optimization Modeling via Dual-Side Verificationcited by 5
  9. 2026MARTI-MARS2: Scaling Multi-Agent Self-Search via Reinforcement Learning for Code Generationcited by 3
  10. 2026ICRL: Learning to Internalize Self-Critique with Reinforcement Learningcited by 3
  11. 2026Rollout-Training Co-Design for Efficient LLM-Based Multi-Agent Reinforcement Learningcited by 2
  12. 2026Modularized Reinforcement Learning on LLMs: From MDP Creation to Exploration and Learningcited by 2
13 more
  1. 2026MoRSE: Task-Oriented Multi-Agent System with Mixture of Role-Subtask Expertscited by 2
  2. 2026Tree-based Credit Assignment for Multi-Agent Memory Systemcited by 1
  3. 2026Experience Sharing in Mutual Reinforcement Learning for Heterogeneous Language Modelscited by 1
  4. 2026MetaAgent-X : Breaking the Ceiling of Automatic Multi-Agent Systems via End-to-End Reinforcement Learningcited by 1
  5. 2026AstraFlow: Dataflow-Oriented Reinforcement Learning for Agentic LLMscited by 1
  6. 2026Reward Modeling for Multi-Agent Orchestrationcited by 1
  7. 2026MetaResearcher: Scaling Deep Research via Self-Reflective Reinforcement Learning in Adversarial Virtual Environmentscited by 1
  8. 2026Can We Steer the Black-Box? Towards Controllability-Centric Evaluation of Recommender Systems with Collaborative Agentscited by 0
  9. 2026SRPO: Setwise Relative Policy Optimization for Multi-Agent LLMscited by 0
  10. 2026CERA-MoA: Co-Evolving Routing Mechanisms with Continually Learning LLM Agentscited by 0
  11. 2026Improving LLM Collaboration via Multi-Agent Preference Learningcited by 0
  12. 2026It Takes Workflows to Evolve Better Workflowscited by 0
  13. 2026Flowing Faster to Coordinate: One-Step Online Multi-Agent Flow Policiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Dr. MAS: Stable Reinforcement Learning for Multi-Agent LLM Systems
  2. 2026MASPO: Joint Prompt Optimization for LLM-based Multi-Agent Systems
  3. 2025Multi-Agent Deep Research: Training Multi-Agent Systems with M-GRPO
  4. 2025Reinforcement Learning-Augmented LLM Agents for Collaborative Decision Making and Performance Optimization
  5. 2025Multi-Agent Tool-Integrated Policy Optimization
  6. 2025Training-Free Group Relative Policy Optimization
  7. 2025LLM Collaboration With Multi-Agent Reinforcement Learning
  8. 2026Enhancing LLM-based Search Agents via Contribution Weighted Group Relative Policy Optimization