mlsanityData to 14 Oct 2026

MALLM: Multi-Agent Large Language Models Framework

arXiv 2509.11656 ↗First posted 15 Sep 2025cs.MA cs.AI cs.CLcited by 15

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Multi-agent debate (MAD) has demonstrated the ability to augment collective intelligence by scaling test-time compute and leveraging expertise. Current frameworks for multi-agent debate are often designed towards tool use, lack integrated evaluation, or provide limited configurability of agent personas, response generators, discussion paradigms, and decision protocols. We introduce MALLM (Multi-Agent Large Language Models), an open-source framework that enables systematic analysis of MAD components. MALLM offers more than 144 unique configurations of MAD, including (1) agent personas (e.g., Expert, Personality), (2) response generators (e.g., Critical, Reasoning), (3) discussion paradigms (e.g., Memory, Relay), and (4) decision protocols (e.g., Voting, Consensus). MALLM uses simple configuration files to define a debate. Furthermore, MALLM can load any textual Hugging Face dataset (e.g., MMLU-Pro, WinoGrande) and provides an evaluation pipeline for easy comparison of MAD configurations. MALLM enables researchers to systematically configure, run, and evaluate debates for their problems, facilitating the understanding of the components and their interplay.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 37 of its 58 references that are in the map. First posted on arXiv 15 Sep 2025.

Builds on

37 of 58 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Revisiting Multi-Agent Debate as Test-Time Scaling: A Systematic Study of Conditional Effectiveness29 May
  2. 2024Improving Multi-Agent Debate with Sparse Communication Topology17 Jun
  3. 2024Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?28 Feb
  4. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs22 Sep
  5. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate30 May

Other references · 32

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  5. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  6. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  7. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  8. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  9. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  10. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
  11. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  12. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 742
20 more
  1. 2020Aligning AI With Shared Human Valuescited by 723
  2. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 431
  3. 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agentscited by 333
  4. 2023State of What Art? A Call for Multi-Prompt LLM Evaluationcited by 310
  5. 2023Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaborationcited by 251
  6. 2023MuSR: Testing the Limits of Chain-of-thought with Multistep Soft Reasoningcited by 201
  7. 2023Personality Traits in Large Language Modelscited by 169
  8. 2023ExpertPrompting: Instructing Large Language Models to be Distinguished Expertscited by 142
  9. 2024AgentScope: A Flexible yet Robust Multi-Agent Platformcited by 121
  10. 2025Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketchingcited by 117
  11. 2023Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMscited by 114
  12. 2023Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communicationcited by 82
  13. 2025Voting or Consensus? Decision-Making in Multi-Agent Debatecited by 59
  14. 2024LLM Voting: Human Choices and AI Collective Decision Makingcited by 48
  15. 2024Multi-Agent Large Language Models for Conversational Task-Solvingcited by 24
  16. 2025Stay Focused: Problem Drift in Multi-Agent Debatecited by 23
  17. 2024LLMs Simulate Big Five Personality Traits: Further Evidencecited by 22
  18. 2024Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challengescited by 18
  19. 2025Literature Review Of Multi-Agent Debate For Problem-Solvingcited by 15
  20. 2024CiteAssist: A System for Automated Preprint Citation and BibTeX Generationcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 14.

15 papers.

Sort:

By area:

  1. 2026When Is Collective Intelligence a Lottery? Multi-Agent Scaling Laws for Memetic Drift in LLMscited by 6
  2. 2026KV Cache Translation across Heterogeneous Large Language Modelscited by 4
  3. 2026When Truth Is Distributed: Misinformation Derails Collective Fact Recovery in LLM-Based Multi-Agent Systemscited by 4
  4. 2026Misinformation Propagation in Benign Multi-Agent Systemscited by 2
  5. 2025More Agents Improve Math Problem Solving but Adversarial Robustness Gap Persistscited by 1
  6. 2026Mind the Gap Between Spatial Reasoning and Acting! Step-by-Step Evaluation of Agents With Spatial-Gymcited by 1
  7. 2026ADEMA: A Knowledge-State Orchestration Architecture for Long-Horizon Knowledge Synthesis with LLMAgentscited by 1
  8. 2026MANTA: Multi-Agent Network Topology Adaptation for Self-Evolving Multi-Agent Systemscited by 1
  9. 2026Creative Generation via Multi-Agent Debate: Does Debate Suppress Diversity?cited by 1
  10. 2026Beyond Sentiment: A Multi-Agent Pipeline for Actionable Business Advice from Reviewscited by 0
  11. 2026L-MAD: A Systematic Evaluation of Multi-Agent Debate Structures in Legal Reasoningcited by 0
  12. 2026Digital Pantheon: Simulating and Auditing Coalition Formation with LLM Agentscited by 0
3 more
  1. 2026Eco3S: Complex Socio-Economic System Simulation via Agent-Based Modelscited by 0
  2. 2026Can Agents Deceive? Evaluating Reasoning and Deception in ParliamentBench using a Social Deduction Gamecited by 0
  3. 2026MABPD: Multi-Agent Bias Probing & Detection via Structured Argument Debatecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Free-MAD: Consensus-Free Multi-Agent Debate
  2. 2025Stop Overvaluing Multi-Agent Debate -- We Must Rethink Evaluation and Embrace Model Heterogeneity
  3. 2026Multi-Agent Debate Strategies: Survey, Taxonomy, and Challenges
  4. 2026M3MAD-Bench: Multi-Dimensional Evaluation of Multi-Agent Debate Across Domains and Modalities
  5. 2026Demystifying Multi-Agent Debate: The Role of Confidence and Diversity
  6. 2026MADBench: Benchmarking the Security of Multi-Agent Debate
  7. 2024Diversity of Thought Elicits Stronger Reasoning Capabilities in Multi-Agent Debate Frameworks
  8. 2025Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models?