XToM: Exploring the Multilingual Theory of Mind for Large Language Models
Not on the map. Why?
Abstract (arXiv)
Theory of Mind (ToM), the ability to infer mental states in others, is pivotal for human social cognition. Existing evaluations of ToM in LLMs are largely limited to English, neglecting the linguistic diversity that shapes human cognition. This limitation raises a critical question: can LLMs exhibit Multilingual Theory of Mind, which is the capacity to reason about mental states across diverse linguistic contexts? To address this gap, we present XToM, a rigorously validated multilingual benchmark that evaluates ToM across five languages and incorporates diverse, contextually rich task scenarios. Using XToM, we systematically evaluate LLMs (e.g., DeepSeek R1), revealing a pronounced dissonance: while models excel in multilingual language understanding, their ToM performance varies across languages. Our findings expose limitations in LLMs' ability to replicate human-like mentalizing across linguistic contexts.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 51 of its 109 references that are in the map. First posted on arXiv 3 Jun 2025.
Builds on
51 of 109 references.
References · 51
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2017Deep reinforcement learning from human preferencescited by 3,846
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
- 2019How multilingual is Multilingual BERT?cited by 642
- 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
- 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
- 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 350
- 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 333
- 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
38 more
- 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 288
- 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
- 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 275
- 2023Mathematical Capabilities of ChatGPTcited by 263
- 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
- 2023Time Travel in LLMs: Tracing Data Contamination in Large Language Modelscited by 175
- 2023Understanding Social Reasoning in Language Models with Language Modelscited by 166
- 2023FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactionscited by 162
- 2023Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Modelscited by 161
- 2023Task Contamination: Language Models May Not Be Few-Shot Anymorecited by 98
- 2023Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languagescited by 96
- 2022ChatGPT: The End of Online Exam Integrity?cited by 88
- 2024MMToM-QA: Multimodal Theory of Mind Question Answeringcited by 84
- 2018Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddingscited by 83
- 2023How FaR Are Large Language Models From Agents with Theory-of-Mind?cited by 75
- 2024MAPO: Advancing Multilingual Reasoning through Multilingual Alignment-as-Preference Optimizationcited by 68
- 2024NegotiationToM: A Benchmark for Stress-testing Machine Theory of Mind on Negotiation Surroundingcited by 49
- 2024Separating Tongue from Thought: Activation Patching Reveals Language-Agnostic Concept Representations in Transformerscited by 42
- 2020Improving Dialog Systems for Negotiation with Personality Modelingcited by 37
- 2023PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Modelscited by 35
- 2024Text-Tuple-Table: Towards Information Integration in Text-to-Table Generation via Global Tuple Extractioncited by 33
- 2024"You Gotta be a Doctor, Lin": An Investigation of Name-Based Bias of Large Language Models in Employment Recommendationscited by 33
- 2021Few-shot Language Coordination by Modeling Theory of Mindcited by 32
- 2023ToMChallenges: A Principle-Guided Dataset and Diverse Evaluation Tasks for Exploring Theory of Mindcited by 28
- 2024CANDLE: Iterative Conceptualization and Instantiation Distillation from Large Language Models for Commonsense Reasoningcited by 28
- 2023StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understandingcited by 26
- 2023DiscoPrompt: Path Prediction Prompt Tuning for Implicit Discourse Relation Recognitioncited by 23
- 2022I Cast Detect Thoughts: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragonscited by 22
- 2024Cultural Conditioning or Placebo? On the Effectiveness of Socio-Demographic Promptingcited by 22
- 2024Converging to a Lingua Franca: Evolution of Linguistic Regions and Semantics Alignment in Multilingual Large Language Modelscited by 22
- 2024Evaluating and Enhancing LLMs Agent based on Theory of Mind in Guandan: A Multi-Player Cooperative Game under Imperfect Informationcited by 20
- 2024ECon: On the Detection and Resolution of Evidence Conflictscited by 18
- 2024Multilingual Brain Surgeon: Large Language Models Can be Compressed Leaving No Language Behindcited by 10
- 2024Investigating Length Issues in Document-level Machine Translationcited by 10
- 2024EventGround: Narrative Reasoning by Grounding to Eventuality-centric Knowledge Graphscited by 9
- 2023Improving Long Context Document-Level Machine Translationcited by 8
- 2025INFERENCEDYNAMICS: Efficient Routing Across LLMs through Structured Capability and Knowledge Profilingcited by 8
- 2024Multi-ToM: Evaluating Multilingual Theory of Mind Capabilities in Large Language Modelscited by 5
50 most cited of 51 All 51 (JSON)
Built on by
Citing papers per year: 2025: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026CoMMET: A Psychologically Grounded Benchmark for Evaluating Theory of Mind in Multimodal LLMs
- 2025From Black Boxes to Transparent Minds: Evaluating and Enhancing the Theory of Mind in Multimodal Large Language Models
- 2024Brittle Minds, Fixable Activations: Understanding Belief Representations in Language Models
- 2023HI-TOM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Models
- 2024Probing the Robustness of Theory of Mind in Large Language Models
- 2023Think Twice: Perspective-Taking Improves Large Language Models' Theory-of-Mind Capabilities
- 2026Language Statistics and False Belief Reasoning: Evidence from 41 Open-Weight LMs
- 2025Do Theory of Mind Benchmarks Need Explicit Human-like Reasoning in Language Models?