mlsanityData to 14 Oct 2026

Mind or Message? Auditing Theory of Mind in Multi-Agent Social Simulation

arXiv 2609.24146 ↗First posted 21 Sep 2026cs.LGcited by 0

On the map: LLM agents and tool use → Social simulation and multi-agent debate

Abstract (arXiv)

Language model agents are increasingly used to simulate social interaction, and the resulting transcripts read as though the agents understand one another. We ask whether that appearance rests on a model of the partner's mind or on the surface record of what the partner said. We build a social simulation in which both questions have exact answers: 40 multi-issue negotiations whose hidden preference weights and whose full Pareto frontier are known by construction. Two model families negotiate across 160 dyads, every transcript is frozen before any measurement, and 2880 counterfactual probes then hold the evidence byte identical while moving one factor at a time: the reader's own stake, the partner's tone, an identity label, and the order of recursion. The agents are socially fluent and economically poor. They reach agreement in 96.2% of dyads with 0 protocol failures, yet only 0.7% of deals land on the Pareto frontier, they leave 20.5% of the available joint value unclaimed, and they miss the one issue on which their interests are perfectly aligned in 76.6% of deals; on the frontier and on that aligned issue, a package drawn at random from the set both sides would accept does as well. The probes locate the failure. Swapping only the reader's own payoff sheet, while the partner's words and offers stay identical, moves the inferred top priority by 15.0 percentage points, which is egocentric projection rather than inference, while a tone rewrite moves it by 5.3 percentage points and an identity label by 0.0. Most tellingly, an agent predicts what its partner believes about it 72.5% of the time while that partner's belief is itself correct only 51.2% of the time: the agents track the conversation far better than they track the mind behind it.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 21 of its 40 references that are in the map. First posted on arXiv 21 Sep 2026.

Builds on

21 of 40 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024AgentsCourt: Building Judicial Decision-Making Agents with Court Debate Simulation and Legal Knowledge Augmentation5 Mar
  2. 2023Generative Agents: Interactive Simulacra of Human Behavior7 Apr

Other references · 19

  1. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  2. 2023Whose Opinions Do Language Models Reflect?cited by 773
  3. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
  4. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 515
  5. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 367
  6. 2023Evaluating Large Language Models in Theory of Mind Taskscited by 222
  7. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
  8. 2017Deal or No Deal? End-to-End Learning for Negotiation Dialoguescited by 209
  9. 2023Understanding Social Reasoning in Language Models with Language Modelscited by 172
  10. 2023FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactionscited by 171
  11. 2023Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Modelscited by 163
  12. 2018Decoupling Strategy and Generation in Negotiation Dialoguescited by 162
7 more
  1. 2023CoMPosT: Characterizing and Evaluating Caricature in LLM Simulationscited by 97
  2. 2023Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiationcited by 96
  3. 2025MARS-SQL: A multi-agent reinforcement learning framework for Text-to-SQLcited by 15
  4. 2025MATP-BENCH: Can MLLM Be a Good Automated Theorem Prover for Multimodal Problems?cited by 13
  5. 2025ClinTutor-R1: Advancing Scalable and Robust One-to-Many Alignment in Clinical Socratic Educationcited by 6
  6. 2026RebuttalAgent: Strategic Persuasion in Academic Rebuttal via Theory of Mindcited by 6
  7. 2026On Stable Long-Form Generation: Benchmarking and Mitigating Length Volatilitycited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Sense and Sensitivity: Evaluating the simulation of social dynamics via Large Language Models
  2. 2026Diversity Without Fidelity: A Solver-Sampler Mismatch in Multi-Agent LLM Negotiation Simulation
  3. 2025Are LLM Agents Behaviorally Coherent? Latent Profiles for Social Simulation
  4. 2026Benchmarking large language model agent societies against human behavioural distributions
  5. 2026The Bystander Effect in Multi-Agent Reasoning: Quantifying Cognitive Loafing in Collaborative Interactions
  6. 2026Simulated Customers Never Walk Away: Decision Fidelity of LLM User Simulators Measured Against Real Purchase Outcomes
  7. 2025The Social Laboratory: A Psychometric Framework for Multi-Agent LLM Evaluation
  8. 2024Simulating Human Strategic Behavior: Comparing Single and Multi-agent LLMs