mlsanityData to 7 Oct 2026

Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View

arXiv 2310.02124 ↗First posted 3 Oct 2023cs.CL cs.AI cs.CYcited by 221

On the map: World models → World models for language agents

also in 1 more
Abstract (arXiv)

As Natural Language Processing (NLP) systems are increasingly employed in intricate social environments, a pressing query emerges: Can these NLP systems mirror human-esque collaborative intelligence, in a multi-agent society consisting of multiple large language models (LLMs)? This paper probes the collaboration mechanisms among contemporary NLP systems by melding practical experiments with theoretical insights. We fabricate four unique `societies' comprised of LLM agents, where each agent is characterized by a specific `trait' (easy-going or overconfident) and engages in collaboration with a distinct `thinking pattern' (debate or reflection). Through evaluating these multi-agent societies on three benchmark datasets, we discern that certain collaborative strategies not only outshine previous top-tier approaches, but also optimize efficiency (using fewer API tokens). Moreover, our results further illustrate that LLM agents manifest human-like social behaviors, such as conformity and consensus reaching, mirroring foundational social psychology theories. In conclusion, we integrate insights from social psychology to contextualize the collaboration of LLM agents, inspiring further investigations into the collaboration mechanism for LLMs. We commit to sharing our code and datasetsfootnoteurlhttps://github.com/zjunlp/MachineSoM., hoping to catalyze further research in this promising avenue.

Key facts. Cited by 221 papers in the map (data through 7 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 29 of its 68 references that are in the map. First posted on arXiv 3 Oct 2023.

Builds on

29 of 68 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Generative Agents: Interactive Simulacra of Human Behavior7 Apr
  2. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs24 Oct

Other references · 27

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  7. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  8. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  9. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  10. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  11. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  12. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
15 more
  1. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  2. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  3. 2023A Survey on Multimodal Large Language Modelscited by 898
  4. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 580
  5. 2023A Survey of Hallucination in Large Foundation Modelscited by 365
  6. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 297
  7. 2023Playing repeated games with Large Language Modelscited by 213
  8. 2021KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extractioncited by 180
  9. 2023Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Modelscited by 161
  10. 2023Agents: An Open-source Framework for Autonomous Language Agentscited by 116
  11. 2023Mindstorms in Natural Language-Based Societies of Mindcited by 105
  12. 2023LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunitiescited by 97
  13. 2023BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agentscited by 83
  14. 2023ChatLLM Network: More brains, More intelligencecited by 37
  15. 2023Interactive Natural Language Processingcited by 31

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 71; 2025: 92; 2026: 51.

221 papers.

Sort:

By area:

  1. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 752
  2. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 424
  3. 2024Mixture-of-Agents Enhances Large Language Model Capabilitiescited by 420
  4. 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 253
  5. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
  6. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 206
  7. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
  8. 2024Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaborationcited by 187
  9. 2024More Agents Is All You Needcited by 174
  10. 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 170
  11. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
  12. 2024Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systemscited by 129
38 more
  1. 2024Simulating Classroom Education with LLM-Empowered Agentscited by 125
  2. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  3. 2024Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agentscited by 120
  4. 2024LLM Multi-Agent Systems: Challenges and Open Problemscited by 119
  5. 2025Red-Teaming LLM Multi-Agent Systems via Communication Attackscited by 118
  6. 2024FinCon: A Synthesized LLM Multi-Agent System with Conceptual Verbal Reinforcement for Enhanced Financial Decision Makingcited by 112
  7. 2024G-Designer: Architecting Multi-agent Communication Topologies via Graph Neural Networkscited by 110
  8. 2025Multi-Agent Design: Optimizing Agents with Better Prompts and Topologiescited by 106
  9. 2025Agentic Large Language Models, a surveycited by 105
  10. 2024Best Practices and Lessons Learned on Synthetic Datacited by 104
  11. 2024Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Modelscited by 97
  12. 2024PersonaGym: Evaluating Persona Agents and LLMscited by 89
  13. 2024Embodied LLM Agents Learn to Cooperate in Organized Teamscited by 81
  14. 2025MasRouter: Learning to Route LLMs for Multi-Agent Systemscited by 80
  15. 2024From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agentscited by 76
  16. 2023CompeteAI: Understanding the Competition Dynamics in Large Language Model-based Agentscited by 65
  17. 2024Many Heads Are Better Than One: Improved Scientific Idea Generation by A LLM-Based Multi-Agent Systemcited by 63
  18. 2024Can Large Language Model Agents Simulate Human Trust Behavior?cited by 62
  19. 2024Agents' Room: Narrative Generation through Multi-step Collaborationcited by 57
  20. 2024Determinants of LLM-assisted Decision-Makingcited by 55
  21. 2024MultiAgent Collaboration Attack: Investigating Adversarial Attacks in Large Language Model Collaborations via Debatecited by 54
  22. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 50
  23. 2025EvoFlow: Evolving Diverse Agentic Workflows On The Flycited by 49
  24. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 49
  25. 2023Towards Reasoning in Large Language Models via Multi-Agent Peer Review Collaborationcited by 48
  26. 2024Shall We Team Up: Exploring Spontaneous Cooperation of Competing LLM Agentscited by 48
  27. 2024Self-Alignment of Large Language Models via Monopolylogue-based Social Scene Simulationcited by 46
  28. 2024Project Sid: Many-agent simulations toward AI civilizationcited by 46
  29. 2025Stop Overvaluing Multi-Agent Debate -- We Must Rethink Evaluation and Embrace Model Heterogeneitycited by 43
  30. 2024Understanding the Dark Side of LLMs' Intrinsic Self-Correctioncited by 42
  31. 2024Large Language Model for Participatory Urban Planningcited by 40
  32. 2025Understanding the Information Propagation Effects of Communication Topologies in LLM-based Multi-Agent Systemscited by 39
  33. 2025GUARDIAN: Safeguarding LLM Multi-Agent Collaborations with Temporal Graph Modelingcited by 35
  34. 2023ALYMPICS: LLM Agents Meet Game Theory -- Exploring Strategic Decision-Making with AI Agentscited by 33
  35. 2026AttriGuard: Defeating Indirect Prompt Injection in LLM Agents via Causal Attribution of Tool Invocationscited by 33
  36. 2024Conformity in Large Language Modelscited by 30
  37. 2024NetSafe: Exploring the Topological Safety of Multi-agent Networkscited by 29
  38. 2025The Truth Becomes Clearer Through Debate! Multi-Agent Systems with Large Language Models Unmask Fake Newscited by 29

50 most cited of 221 All 221 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025The Social Laboratory: A Psychometric Framework for Multi-Agent LLM Evaluation
  2. 2026"Who Am I, and Who Else Is Here?" Behavioral Differentiation Without Role Assignment in Multi-Agent LLM Systems
  3. 2025Beyond Static Responses: Multi-Agent LLM Systems as a New Paradigm for Social Science Research
  4. 2025Do as We Do, Not as You Think: the Conformity of Large Language Models
  5. 2024Harnessing the power of LLMs for normative reasoning in MASs
  6. 2025Through the Lens of Human-Human Collaboration: A Configurable Research Platform for Exploring Human-Agent Collaboration
  7. 2025Exploring Large Language Model Agents for Piloting Social Experiments
  8. 2025Collaborate, Deliberate, Evaluate: How LLM Alignment Affects Coordinated Multi-Agent Outcomes