mlsanityData to 14 Oct 2026

AutoToM: Scaling Model-based Mental Inference via Automated Agent Modeling

arXiv 2502.15676 ↗First posted 21 Feb 2025cs.AI cs.CLcited by 23

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Theory of Mind (ToM), the ability to understand people's minds based on their behavior, is key to developing socially intelligent agents. Current approaches to ToM reasoning either rely on prompting Large Language Models (LLMs), which are prone to systematic errors, or use handcrafted, rigid agent models for model-based inference, which are more robust but fail to generalize across domains. In this work, we introduce AutoToM, an automated agent modeling method for scalable, robust, and interpretable mental inference. Given a ToM problem, AutoToM first proposes an initial agent model and then performs automated Bayesian inverse planning based on this model, leveraging an LLM backend. Guided by inference uncertainty, it iteratively refines the model by introducing additional mental variables and/or incorporating more timesteps in the context. Across five diverse benchmarks, AutoToM outperforms existing ToM methods and even large reasoning models. Additionally, we show that AutoToM can produce human-like confidence estimates and enable online mental inference for embodied decision-making.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 28 of its 49 references that are in the map. First posted on arXiv 21 Feb 2025.

Builds on

28 of 49 references.

References · 28

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2016Cooperative Inverse Reinforcement Learningcited by 300
  3. 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 292
  4. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
  5. 2023Understanding Social Reasoning in Language Models with Language Modelscited by 172
  6. 2023FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactionscited by 171
  7. 2023Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Modelscited by 163
  8. 2023Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Trackercited by 95
  9. 2023Think Twice: Perspective-Taking Improves Large Language Models' Theory-of-Mind Capabilitiescited by 89
  10. 2024MMToM-QA: Multimodal Theory of Mind Question Answeringcited by 85
  11. 2020Online Bayesian Goal Inference for Boundedly-Rational Planning Agentscited by 68
  12. 2023HI-TOM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Modelscited by 63
16 more
  1. 2021AGENT: A Benchmark for Core Psychological Reasoningcited by 42
  2. 2024Automated Statistical Model Discovery with Language Modelscited by 41
  3. 2025Hypothesis-Driven Theory-of-Mind Reasoning for Large Language Modelscited by 29
  4. 2024Pragmatic Instruction Following and Goal Assistance via Cooperative Language-Guided Inverse Planningcited by 28
  5. 2024Doing Experiments and Revising Rules with Natural Language and Probabilistic Reasoningcited by 24
  6. 2016Deep Amortized Inference for Probabilistic Programscited by 23
  7. 2023NOPA: Neurally-guided Online Probabilistic Assistance for Building Socially Intelligent Home Assistantscited by 23
  8. 2021PHASE: PHysically-grounded Abstract Social Events for Machine Social Perceptioncited by 22
  9. 2024TimeToM: Temporal Space is the Key to Unlocking the Door of Large Language Models' Theory-of-Mindcited by 21
  10. 2023HandMeThat: Human-Robot Communication in Physical and Social Environmentscited by 20
  11. 2022Proactive Robot Assistance via Spatio-Temporal Object Modelingcited by 18
  12. 2024GOMA: Proactive Embodied Cooperative Communication via Goal-Oriented Mental Alignmentcited by 16
  13. 2024A Notion of Complexity for Theory of Mind via Discrete World Modelscited by 16
  14. 2020StylePredict: Machine Theory of Mind for Human Driver Behavior From Trajectoriescited by 11
  15. 2018Goal Inference Improves Objective and Perceived Performance in Human-Robot Collaborationcited by 10
  16. 2023Neural Amortized Inference for Nested Multi-agent Reasoningcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 12.

23 papers.

Sort:

By area:

  1. 2025Modeling Open-World Cognition as On-Demand Synthesis of Probabilistic Modelscited by 14
  2. 2025ToMAP: Training Opponent-Aware LLM Persuaders with Theory of Mindcited by 12
  3. 2025Do Large Language Models Have a Planning Theory of Mind? Evidence from MindGames: a Multi-Step Persuasion Taskcited by 9
  4. 2025Modeling Others' Minds as Codecited by 7
  5. 2026ThoughtTrace: Understanding User Thoughts in Real-World LLM Interactionscited by 7
  6. 2026AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnessescited by 7
  7. 2025TactfulToM: Do LLMs Have the Theory of Mind Ability to Understand White Lies?cited by 5
  8. 2025MindPower: Enabling Theory-of-Mind Reasoning in VLM-based Embodied Agentscited by 5
  9. 2026From Shortcuts to Reasoning: Robust Post-Training of Theory of Mind with Reinforcement Learningcited by 2
  10. 2025BeliefNest: A Joint Action Simulator for Embodied Agents with Theory of Mindcited by 1
  11. 2025Mind the Motions: Benchmarking Theory-of-Mind in Everyday Body Languagecited by 1
  12. 2025RecToM: A Benchmark for Evaluating Machine Theory of Mind in LLM-based Conversational Recommender Systemscited by 1
11 more
  1. 2025A Computable Game-Theoretic Framework for Multi-Agent Theory of Mindcited by 1
  2. 2026Grounding Social Perception in Intuitive Physicscited by 1
  3. 2026PDDL-Mind: Large Language Models are Capable on Belief Reasoning with Reliable State Trackingcited by 1
  4. 2026UserHarness: Harnessing User Minds for Stronger Agent Theory-of-Mindcited by 1
  5. 2026Hypotheses-Guided Self Distillation for Continual Personalizationcited by 1
  6. 2026Mind2Dialogue: Training Human-Aware Language Models by Simulating User Mental Statescited by 1
  7. 2025TimeHC-RL: Temporal-aware Hierarchical Cognitive Reinforcement Learning for Enhancing LLMs' Social Intelligencecited by 0
  8. 2026MindZero: Learning Online Mental Reasoning With Zero Annotationscited by 0
  9. 2026Resonant Minds: Closed-Loop Social Avatars with Theory of Mindcited by 0
  10. 2026RevengeBench: Reverse Engineering Code-Space Policies from Behavioral Experimentscited by 0
  11. 2026OverForge: Reasoning Through Strategies and Tactics Helps Cooperative Lifelong Adaptationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Machine Theory of Mind with Large Language Model-Augmented Inverse Planning
  2. 2025Overcoming Multi-step Complexity in Multimodal Theory-of-Mind Reasoning: A Scalable Bayesian Planner
  3. 2025Theory of Mind in Action: The Instruction Inference Task in Dynamic Human-Agent Collaboration
  4. 2024OpenToM: A Comprehensive Benchmark for Evaluating Theory-of-Mind Reasoning Capabilities of Large Language Models
  5. 2024Mental Modeling of Reinforcement Learning Agents by Language Models
  6. 2025Decompose-ToM: Enhancing Theory of Mind Reasoning in Large Language Models through Simulation and Task Decomposition
  7. 2024SimpleToM: Exposing the Gap between Explicit ToM Inference and Implicit ToM Application in LLMs
  8. 2026Latent Perspective-Taking via a Schr"odinger Bridge in Influence-Augmented Local Models ↗