mlsanityData to 14 Oct 2026

Avalon-ToM-Bench: Evaluating Fine-Grained Theory of Mind via Asymmetric Game Mechanics

arXiv 2608.09638 ↗First posted 10 Aug 2026cs.AI cs.CL cs.CYcited by 0

Not on the map. Why?

Abstract (arXiv)

Theory of Mind (ToM) is essential for agent interactions, yet existing evaluations either rely on static scenarios that oversimplify mental-state reasoning or interactive settings that provide limited diagnostic insight. We present Avalon-ToM-Bench, a fine-grained benchmark that operationalizes ToM through the asymmetric-information mechanics of The Resistance: Avalon. Rather than evaluating end-to-end gameplay, it decomposes ToM into a 2×2 taxonomy -- epistemic versus motivational reasoning crossed with inference versus action -- using human-crafted, perspective-constrained queries. Benchmarking 28 LLMs reveals three insights: 1) Reasoning, not knowledge. Models show strong game-rule comprehension but markedly weaker ToM abilities, isolating failures to social reasoning rather than missing domain knowledge. 2) Expression, not representation. Mechanistic analyses via linear probing and activation steering show that models frequently represent correct mental-state inferences in their hidden states but fail to express them during generation -- linear probes recover 77-82% accuracy versus 62-70% from the models' own chain-of-thought. 3) Policy, not deliberation. Dedicated reasoning training yields substantial improvements whereas test-time chain-of-thought provides only marginal gains (+11.0 versus +1.1 points on average), suggesting that robust ToM depends on a learned reasoning policy rather than increased inference-time deliberation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 24 references that are in the map. First posted on arXiv 10 Aug 2026.

Builds on

15 of 24 references.

References · 15

  1. 2023Steering Language Models With Activation Engineeringcited by 922
  2. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  3. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  4. 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 292
  5. 2023Exploring Large Language Models for Communication Games: An Empirical Study on Werewolfcited by 241
  6. 2023Understanding Social Reasoning in Language Models with Language Modelscited by 172
  7. 2023FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactionscited by 171
  8. 2023Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Modelscited by 163
  9. 2023AvalonBench: Evaluating LLMs Playing the Game of Avaloncited by 47
  10. 2019Finding Friend and Foe in Multi-Agent Gamescited by 42
  11. 2025ToMATO: Verbalizing the Mental States of Role-Playing LLMs for Benchmarking Theory of Mindcited by 21
  12. 2024Strategist: Self-improvement of LLM Decision Making via Bi-Level Tree Searchcited by 17
3 more
  1. 2025Beyond Survival: Evaluating LLMs in Social Deduction Games with Human-Aligned Strategiescited by 10
  2. 2025Bayesian Social Deduction with Graph-Informed Language Modelscited by 9
  3. 2025InMind: Evaluating LLMs in Capturing and Applying Individual Human Reasoning Stylescited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025The Decrypto Benchmark for Multi-Agent Reasoning and Theory of Mind
  2. 2026Readable Minds: Emergent Theory-of-Mind-Like Behavior in LLM Poker Agents
  3. 2026MINDGAMES: A Live Arena for Evaluating Social and Strategic Reasoning in Multi-Agent LLMs
  4. 2025Do Large Language Models Have a Planning Theory of Mind? Evidence from MindGames: a Multi-Step Persuasion Task
  5. 2025LLM-Hanabi: Evaluating Multi-Agent Gameplays with Theory-of-Mind and Rationale Inference in Imperfect Information Collaboration Game
  6. 2026Beyond Sally-Anne: Evaluating Theory of Mind in LLMs using Epistemic Schelling Points
  7. 2026Adaptive Theory of Mind for LLM-based Multi-Agent Coordination
  8. 2026Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models