mlsanityData to 14 Oct 2026

ACC-Collab: An Actor-Critic Approach to Multi-Agent LLM Collaboration

arXiv 2411.00053 ↗First posted 30 Oct 2024cs.CL cs.AIcited by 24

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Large language models (LLMs) have demonstrated a remarkable ability to serve as general-purpose tools for various language-based tasks. Recent works have demonstrated that the efficacy of such models can be improved through iterative dialog between multiple models. While these paradigms show promise in improving model efficacy, most works in this area treat collaboration as an emergent behavior, rather than a learned behavior. In doing so, current multi-agent frameworks rely on collaborative behaviors to have been sufficiently trained into off-the-shelf models. To address this limitation, we propose ACC-Collab, an Actor-Critic based learning framework to produce a two-agent team (an actor-agent and a critic-agent) specialized in collaboration. We demonstrate that ACC-Collab outperforms SotA multi-agent techniques on a wide array of benchmarks.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 49 of its 63 references that are in the map. First posted on arXiv 30 Oct 2024.

Builds on

49 of 63 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024Improving Multi-Agent Debate with Sparse Communication Topology17 Jun
  2. 2024Watch Every Step! LLM Agent Learning via Iterative Step-Level Process Refinement17 Jun
  3. 2024Confidence Calibration and Rationalization for LLMs via Multi-Agent Deliberation14 Apr
  4. 2024Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?28 Feb
  5. 2024More Agents Is All You Need3 Feb
  6. 2024LLM Harmony: Multi-Agent Communication for Problem Solving2 Jan
  7. 2023Theory of Mind for Multi-Agent Collaboration via Large Language Models16 Oct
  8. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate14 Aug
  9. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework1 Aug
  10. 2023Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration11 Jul

Other references · 39

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  7. 2023Mistral 7Bcited by 3,164
  8. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  9. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  10. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  11. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  12. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
27 more
  1. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 1,011
  2. 2017Crowdsourcing Multiple Choice Science Questionscited by 765
  3. 2018AI safety via debatecited by 400
  4. 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 384
  5. 2023A Survey of Hallucination in Large Foundation Modelscited by 368
  6. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 309
  7. 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 269
  8. 2024Debating with More Persuasive LLMs Leads to More Truthful Answerscited by 266
  9. 2024Iterative Reasoning Preference Optimizationcited by 246
  10. 2024ReFT: Reasoning with Reinforced Fine-Tuningcited by 239
  11. 2023Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology Viewcited by 222
  12. 2024Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaborationcited by 180
  13. 2023Multilingual Machine Translation with Large Language Models: Empirical Results and Analysiscited by 160
  14. 2023PRD: Peer Rank and Discussion Improve Large Language Model based Evaluationscited by 114
  15. 2023Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMscited by 114
  16. 2024MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUscited by 112
  17. 2024GLoRe: When, Where, and How to Improve LLM Reasoning via Global and Local Refinementscited by 103
  18. 2024LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Modelscited by 99
  19. 2023Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debatecited by 94
  20. 2023MetaAgents: Large Language Model Based Agents for Decision-Making on Teamingcited by 85
  21. 2024LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Modelscited by 70
  22. 2024Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoningcited by 58
  23. 2023Let Models Speak Ciphers: Multiagent Debate through Embeddingscited by 51
  24. 2023Debate Helps Supervise Unreliable Expertscited by 43
  25. 2024Can LLMs Speak For Diverse People? Tuning LLMs via Debate to Generate Controllable Controversial Statementscited by 34
  26. 2019Convergence of Learning Dynamics in Stackelberg Gamescited by 28
  27. 2024SocraSynth: Multi-LLM Reasoning with Conditional Statisticscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 12; 2026: 12.

24 papers.

Sort:

By area:

  1. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 111
  2. 2025Talk Isn't Always Cheap: Understanding Failure Modes in Multi-Agent Debatecited by 66
  3. 2025CoMAS: Co-Evolving Multi-Agent Systems via Interaction Rewardscited by 29
  4. 2025Multi-Agent Debate for LLM Judges with Adaptive Stability Detectioncited by 23
  5. 2025How to Train a Leader: Hierarchical Reasoning in Multi-Agent LLMscited by 12
  6. 2025DEBATE, TRAIN, EVOLVE: Self Evolution of Language Model Reasoningcited by 6
  7. 2025Interactive Learning for LLM Reasoningcited by 5
  8. 2025DART: Leveraging Multi-Agent Disagreement for Tool Recruitment in Multimodal Reasoningcited by 5
  9. 2025Emergent Bias and Fairness in Multi-Agent Decision Systemscited by 5
  10. 2025Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanismcited by 4
  11. 2025MarsRL: Advancing Multi-Agent Reasoning System via Reinforcement Learning with Agentic Pipeline Parallelismcited by 4
  12. 2026Asymmetric Actor-Critic for Multi-turn LLM Agentscited by 4
12 more
  1. 2026Multi-Agent Dialectical Refinement for Enhanced Argument Classificationcited by 3
  2. 2026Hierarchical Reinforcement Learning with Augmented Step-Level Transitions for LLM Agentscited by 2
  3. 2026AgentNLQ: A General-Purpose Agent for Natural Language to SQLcited by 2
  4. 2026From Argument Components to Graphs: A Multi-Agent Debate with Confidence Gating for Argument Relationscited by 1
  5. 2025Debating for Better Reasoning: An Unsupervised Multimodal Approachcited by 0
  6. 2026When Does Critique Improve AI-Assisted Theoretical Physics? SCALAR: Structured Critic--Actor Loop for Agentic Reasoningcited by 0
  7. 2026Iterative Critique-and-Routing Controller for Multi-Agent Systems with Heterogeneous LLMscited by 0
  8. 2026Examining Agents' Bias Amplification versus Suppression in Multi-Agent Systemscited by 0
  9. 2026CAF-Gen: A Multi-Agent System for Enriching Argumentation Structurescited by 0
  10. 2026SEPAL: Separated Expert Pairs with Answer-Level Fusion for Reliable LLM Collaborationcited by 0
  11. 2026Observe, Share, or Predict? Disentangling Partner Information in Embodied Multi-Agent Collaborationcited by 0
  12. 2026Compositional RL: Evaluating and Training LLMs for Collaborative Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Collab-Overcooked: Benchmarking and Evaluating Large Language Models as Collaborative Agents
  2. 2025MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learning
  3. 2026CollabEval: Enhancing LLM-as-a-Judge via Multi-Agent Collaboration
  4. 2026Learning Decentralized LLM Collaboration with Multi-Agent Actor Critic
  5. 2024Towards Efficient LLM Grounding for Embodied Multi-Agent Collaboration
  6. 2024CMAT: A Multi-Agent Collaboration Tuning Framework for Enhancing Small Language Models
  7. 2026CollabSim: A CSCW-Grounded Methodology for Investigating Collaborative Competence of LLM Agents through Controlled Multi-Agent Experiments
  8. 2025Multi-Agent Language Models: Advancing Cooperation, Coordination, and Adaptation