mlsanityData to 14 Oct 2026

MASLab: A Unified and Comprehensive Codebase for LLM-based Multi-Agent Systems

arXiv 2505.16988 ↗First posted 22 May 2025cs.CL cs.AI cs.MAcited by 14

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

LLM-based multi-agent systems (MAS) have demonstrated significant potential in enhancing single LLMs to address complex and diverse tasks in practical applications. Despite considerable advancements, the field lacks a unified codebase that consolidates existing methods, resulting in redundant re-implementation efforts, unfair comparisons, and high entry barriers for researchers. To address these challenges, we introduce MASLab, a unified, comprehensive, and research-friendly codebase for LLM-based MAS. (1) MASLab integrates over 20 established methods across multiple domains, each rigorously validated by comparing step-by-step outputs with its official implementation. (2) MASLab provides a unified environment with various benchmarks for fair comparisons among methods, ensuring consistent inputs and standardized evaluation protocols. (3) MASLab implements methods within a shared streamlined structure, lowering the barriers for understanding and extension. Building on MASLab, we conduct extensive experiments covering 10+ benchmarks and 8 models, offering researchers a clear and comprehensive view of the current landscape of MAS methods. MASLab will continue to evolve, tracking the latest developments in the field, and invite contributions from the broader open-source community.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 51 of its 65 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

51 of 65 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025MAS-GPT: Training LLMs to Build LLM-based Multi-Agent Systems5 Mar
  2. 2024Synthesizing Post-Training Data for LLMs through Multi-Agent Simulation18 Oct
  3. 2024G-Designer: Architecting Multi-agent Communication Topologies via Graph Neural Networks15 Oct
  4. 2024Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systems3 Oct
  5. 2024Automated Design of Agentic Systems15 Aug
  6. 2024Scaling Large Language Model-based Multi-Agent Collaboration11 Jun
  7. 2023OpenAgents: An Open Platform for Language Agents in the Wild16 Oct
  8. 2023A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration3 Oct
  9. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors21 Aug
  10. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework1 Aug
  11. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society31 Mar

Other references · 40

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  8. 2024Qwen2.5 Technical Reportcited by 4,942
  9. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  10. 2021Program Synthesis with Large Language Modelscited by 3,637
  11. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  12. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
28 more
  1. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  2. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  3. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  4. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  5. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  6. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  7. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
  8. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 1,011
  9. 2023BloombergGPT: A Large Language Model for Financecited by 939
  10. 2023ChatDev: Communicative Agents for Software Developmentcited by 919
  11. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  12. 2022PAL: Program-aided Language Modelscited by 652
  13. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 650
  14. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 543
  15. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 503
  16. 2024AFlow: Automating Agentic Workflow Generationcited by 432
  17. 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 425
  18. 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoningcited by 316
  19. 2023Towards Generalist Biomedical AIcited by 289
  20. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 256
  21. 2023MathPrompter: Mathematical Reasoning using Large Language Modelscited by 236
  22. 2024MapCoder: Multi-Agent Code Generation for Competitive Problem Solvingcited by 177
  23. 2023Fundamental Limitations of Alignment in Large Language Modelscited by 155
  24. 2025Multiagent Finetuning: Self Improvement with Diverse Reasoning Chainscited by 95
  25. 2025xVerify: Efficient Answer Verifier for Reasoning Model Evaluationscited by 55
  26. 2024Self-Alignment of Large Language Models via Monopolylogue-based Social Scene Simulationcited by 46
  27. 2024MACM: Utilizing a Multi-Agent System for Condition Mining in Solving Complex Mathematical Problemscited by 41
  28. 2025GNNs as Predictors of Agentic Workflow Performancescited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 9; 2026: 5.

14 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2024A Survey on LLM-based Multi-Agent System: Recent Advances and New Frontiers in Application
  2. 2025X-MAS: Towards Building Multi-Agent Systems with Heterogeneous LLMs
  3. 2025Beyond Self-Talk: A Communication-Centric Survey of LLM-Based Multi-Agent Systems
  4. 2025MasRouter: Learning to Route LLMs for Multi-Agent Systems
  5. 2026LLM-Enabled Multi-Agent Systems: Empirical Evaluation and Insights into Emerging Design Patterns & Paradigms
  6. 2026MASEval: Extending Multi-Agent Evaluation from Models to Systems
  7. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges
  8. 2026LLM-Based Multi-Agent Systems for Code Generation: A Multi-Vocal Literature Review