mlsanityData to 7 Oct 2026

EnactToM: An Evolving Benchmark for Functional Theory of Mind in Embodied Agents

arXiv 2605.09826 ↗First posted 11 May 2026cs.AI cs.MAcited by 3

Not on the map. Why?

Abstract (arXiv)

Theory of Mind (ToM), the ability to track others epistemic state, makes humans efficient collaborators. AI agents need the same capacity in multi agent settings, yet existing benchmarks mostly test literal ToM by asking direct belief questions. The ability act optimally on implicit beliefs in embodied environments, called functional ToM, remains largely untested. We introduce EnactToM, an evolving benchmark of 300 embodied multi-agent tasks set in a 3D household with partial observability, private information, and constrained communication. Each task is formally verified for solvability and required epistemic depth, and new tasks are generated increase difficulty as models improve. On the hard split, all seven evaluated frontier models score 0.0% Pass^3 on functional task completion, while averaging 45.0% on literal belief probes. Manual analysis traces 93% of sampled failures to epistemic coordination breakdowns such as withheld information, ignored partner constraints, and misallocated messages, providing a concrete target for future work.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 31 of its 62 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

31 of 62 references.

References · 31

  1. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  2. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  3. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  4. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  5. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  6. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  7. 2018VirtualHome: Simulating Household Activities via Programscited by 411
  8. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 350
  9. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 345
  10. 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 288
  11. 2024LiveBench: A Challenging, Contamination-Limited LLM Benchmarkcited by 242
  12. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
19 more
  1. 2023Evaluating Large Language Models in Theory of Mind Taskscited by 217
  2. 2019The Hanabi Challenge: A New Frontier for AI Researchcited by 194
  3. 2021TEACh: Task-driven Embodied Agents that Chatcited by 194
  4. 2023Time Travel in LLMs: Tracing Data Contamination in Large Language Modelscited by 175
  5. 2023Understanding Social Reasoning in Language Models with Language Modelscited by 166
  6. 2023FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactionscited by 162
  7. 2023Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Modelscited by 161
  8. 2023Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarkscited by 139
  9. 2024MMToM-QA: Multimodal Theory of Mind Question Answeringcited by 84
  10. 2024PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Taskscited by 78
  11. 2023How FaR Are Large Language Models From Agents with Theory-of-Mind?cited by 75
  12. 2022Mapping global dynamics of benchmark creation and saturation in artificial intelligencecited by 71
  13. 2023HI-TOM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Modelscited by 61
  14. 2024OpenToM: A Comprehensive Benchmark for Evaluating Theory-of-Mind Reasoning Capabilities of Large Language Modelscited by 54
  15. 2024MuMA-ToM: Multi-modal Multi-Agent Theory of Mindcited by 51
  16. 2024SimpleToM: Exposing the Gap between Explicit ToM Inference and Implicit ToM Application in LLMscited by 51
  17. 2024Explore Theory of Mind: Program-guided adversarial data generation for theory of mind reasoningcited by 30
  18. 2024Position: Theory of Mind Benchmarks are Broken for Large Language Modelscited by 30
  19. 2026When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturationcited by 29

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.