mlsanityData to 7 Oct 2026

Mindstorms in Natural Language-Based Societies of Mind

arXiv 2305.17066 ↗First posted 26 May 2023cs.AI cs.CL cs.CVcited by 105

Not on the map. Why?

Cited by 68 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Both Minsky's "society of mind" and Schmidhuber's "learning to think" inspire diverse societies of large multimodal neural networks (NNs) that solve problems by interviewing each other in a "mindstorm." Recent implementations of NN-based societies of minds consist of large language models (LLMs) and other NN-based experts communicating through a natural language interface. In doing so, they overcome the limitations of single LLMs, improving multimodal zero-shot reasoning. In these natural language-based societies of mind (NLSOMs), new agents -- all communicating through the same universal symbolic language -- are easily added in a modular fashion. To demonstrate the power of NLSOMs, we assemble and experiment with several of them (having up to 129 members), leveraging mindstorms in them to solve some practical AI tasks: visual question answering, image captioning, text-to-image synthesis, 3D generation, egocentric retrieval, embodied AI, and general language-based task solving. We view this as a starting point towards much larger NLSOMs with billions of agents-some of which may be humans. And with this emergence of great societies of heterogeneous minds, many new research questions have suddenly become paramount to the future of artificial intelligence. What should be the social structure of an NLSOM? What would be the (dis)advantages of having a monarchical rather than a democratic structure? How can principles of NN economies be used to maximize the total reward of a reinforcement learning NLSOM? In this work, we identify, discuss, and try to answer some of these questions.

Key facts. Cited by 105 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 May 2023.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 15; 2024: 38; 2025: 33; 2026: 19.

105 papers.

Sort:

By area:

  1. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  2. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  3. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 956
  4. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 617
  5. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 424
  6. 2024AFlow: Automating Agentic Workflow Generationcited by 395
  7. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 297
  8. 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 294
  9. 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 253
  10. 2024Agent-as-a-Judge: Evaluate Agents with Agentscited by 223
  11. 2024Data Interpreter: An LLM Agent For Data Sciencecited by 222
  12. 2023Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology Viewcited by 221
38 more
  1. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  2. 2024Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffoldingcited by 115
  3. 2024Secret Collusion among AI Agents: Multi-Agent Deception via Steganographycited by 107
  4. 2025Agentic Large Language Models, a surveycited by 105
  5. 2023Evil Geniuses: Delving into the Safety of LLM-based Agentscited by 103
  6. 2024Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agentscited by 91
  7. 2024Language Agents as Optimizable Graphscited by 81
  8. 2025ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learningcited by 78
  9. 2025FlowReasoner: Reinforcing Query-Level Meta-Agentscited by 56
  10. 2025ScoreFlow: Mastering LLM Agent Workflows via Score-based Preference Optimizationcited by 50
  11. 2024LLM Voting: Human Choices and AI Collective Decision Makingcited by 47
  12. 2024DetToolChain: A New Prompting Paradigm to Unleash Detection Ability of MLLMcited by 45
  13. 2024Evaluating Language Model Agency through Negotiationscited by 43
  14. 2024Adaptive In-conversation Team Building for Language Model Agentscited by 38
  15. 2023Urban Generative Intelligence (UGI): A Foundational Platform for Agents in Embodied City Environmentcited by 37
  16. 2025Multi-Agent Coordination across Diverse Applications: A Surveycited by 37
  17. 2024Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Modelscited by 34
  18. 2024LLM Agents in Interaction: Measuring Personality Consistency and Linguistic Alignment in Interacting Populations of Large Language Modelscited by 33
  19. 2023AgentCoder: Multi-Agent-based Code Generation with Iterative Testing and Optimisationcited by 31
  20. 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
  21. 2025textitAgents Under Siege: Breaking Pragmatic Multi-Agent LLM Systems with Optimized Prompt Attackscited by 31
  22. 2025Internet of Agents: Fundamentals, Applications, and Challengescited by 31
  23. 2023An In-depth Survey of Large Language Model-based Artificial Intelligence Agentscited by 30
  24. 2024AgentCoord: Visually Exploring Coordination Strategy for LLM-based Multi-Agent Collaborationcited by 26
  25. 2024InterIntent: Investigating Social Intelligence of LLMs via Intention Understanding in an Interactive Game Contextcited by 25
  26. 2024Hierarchical Auto-Organizing System for Open-Ended Multi-Agent Navigationcited by 24
  27. 2024Multi-Agent Large Language Models for Conversational Task-Solvingcited by 24
  28. 2024S-Agents: Self-organizing Agents in Open-ended Environmentscited by 22
  29. 2024ControlAgent: Automating Control System Design via Novel Integration of LLM Agents and Domain Expertisecited by 20
  30. 2024Do We Really Need a Complex Agent System? Distill Embodied Agent into a Single Modelcited by 19
  31. 2025Multi-Agent Image Restorationcited by 19
  32. 2024How to Correctly do Semantic Backpropagation on Language-based Agentic Systemscited by 18
  33. 2024Self-Recognition in Language Modelscited by 17
  34. 2026Self-Improvements in Modern Agentic Systems: A Surveycited by 17
  35. 2024MineLand: Simulating Large-Scale Multi-Agent Interactions with Limited Multimodal Senses and Physical Needscited by 16
  36. 2024LLM-based multi-agent poetry generation in non-cooperative environmentscited by 15
  37. 2025AMAS: Adaptively Determining Communication Topology for LLM-based Multi-Agent Systemcited by 15
  38. 2024Towards Truly Zero-shot Compositional Visual Reasoning with LLMs as Programmerscited by 14

50 most cited of 105 All 105 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Large language models for artificial general intelligence (AGI): A survey of foundational principles and approaches
  2. 2025On the Thinking-Language Modeling Gap in Large Language Models
  3. 2023How FaR Are Large Language Models From Agents with Theory-of-Mind?
  4. 2023Evaluating Large Language Models in Theory of Mind Tasks
  5. 2023Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models
  6. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs
  7. 2023Understanding Social Reasoning in Language Models with Language Models
  8. 2025From System 1 to System 2: A Survey of Reasoning Large Language Models