mlsanityData to 7 Oct 2026

Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models

arXiv 2305.14763 ↗First posted 24 May 2023cs.CLcited by 161

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

The escalating debate on AI's capabilities warrants developing reliable metrics to assess machine "intelligence". Recently, many anecdotal examples were used to suggest that newer large language models (LLMs) like ChatGPT and GPT-4 exhibit Neural Theory-of-Mind (N-ToM); however, prior work reached conflicting conclusions regarding those abilities. We investigate the extent of LLMs' N-ToM through an extensive evaluation on 6 tasks and find that while LLMs exhibit certain N-ToM abilities, this behavior is far from being robust. We further examine the factors impacting performance on N-ToM tasks and discover that LLMs struggle with adversarial examples, indicating reliance on shallow heuristics rather than robust ToM abilities. We caution against drawing conclusions from anecdotal examples, limited benchmark testing, and using human-designed psychological tests to evaluate models.

Key facts. Cited by 161 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 31 of its 74 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

31 of 74 references.

References · 31

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  6. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  7. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  8. 2023A Survey of Large Language Modelscited by 2,537
  9. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  10. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  11. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  12. 2019Natural Adversarial Examplescited by 1,401
19 more
  1. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 481
  2. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  3. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 428
  4. 2015Towards AI-Complete Question Answering: A Set of Prerequisite Toy Taskscited by 309
  5. 2022UL2: Unifying Language Learning Paradigmscited by 292
  6. 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 288
  7. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
  8. 2021Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Modelscited by 219
  9. 2020Unsupervised Commonsense Question Answering with Self-Talkcited by 161
  10. 2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanationscited by 159
  11. 2018Breaking NLI Systems with Sentences that Require Simple Lexical Inferencescited by 134
  12. 2022The 'Problem' of Human Label Variation: On Ground Truth in Data, Modeling and Evaluationcited by 133
  13. 2018Event2Mind: Commonsense Inference on Events, Intents, and Reactionscited by 82
  14. 2022Tomayto, Tomahto. Beyond Token-level Answer Equivalence for Question Answering Evaluationcited by 66
  15. 2018Evaluating Theory of Mind in Question Answeringcited by 64
  16. 2018Modeling Naive Psychology of Characters in Simple Commonsense Storiescited by 59
  17. 2021What Context Features Can Transformer Language Models Use?cited by 49
  18. 2019When Choosing Plausible Alternatives, Clever Hans can be Clevercited by 39
  19. 2021A Semantic-based Method for Unsupervised Commonsense Question Answeringcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 28; 2024: 39; 2025: 51; 2026: 43.

161 papers.

Sort:

By area:

  1. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  2. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 350
  3. 2023Exploring Large Language Models for Communication Games: An Empirical Study on Werewolfcited by 238
  4. 2023Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology Viewcited by 221
  5. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 218
  6. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 206
  7. 2023Understanding Social Reasoning in Language Models with Language Modelscited by 166
  8. 2023Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Trackercited by 95
  9. 2024SOTOPIA-π: Interactive Learning of Socially Intelligent Language Agentscited by 89
  10. 2023Think Twice: Perspective-Taking Improves Large Language Models' Theory-of-Mind Capabilitiescited by 86
  11. 2024MMToM-QA: Multimodal Theory of Mind Question Answeringcited by 84
  12. 2023Put Your Money Where Your Mouth Is: Evaluating Strategic Planning and Execution of LLM Agents in an Auction Arenacited by 78
38 more
  1. 2023How FaR Are Large Language Models From Agents with Theory-of-Mind?cited by 75
  2. 2024ToMBench: Benchmarking Theory of Mind in Large Language Modelscited by 70
  3. 2023Turning large language models into cognitive modelscited by 66
  4. 2023ARB: Advanced Reasoning Benchmark for Large Language Modelscited by 60
  5. 2023Language Models, Agent Models, and World Models: The LAW for Machine Reasoning and Planningcited by 59
  6. 2024LLMs achieve adult human performance on higher-order theory of mind taskscited by 59
  7. 2024SimpleToM: Exposing the Gap between Explicit ToM Inference and Implicit ToM Application in LLMscited by 51
  8. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 50
  9. 2023Lyfe Agents: Generative agents for low-cost real-time social interactionscited by 49
  10. 2024NegotiationToM: A Benchmark for Stress-testing Machine Theory of Mind on Negotiation Surroundingcited by 49
  11. 2024Large Language Models Assume People are More Rational than We Really arecited by 49
  12. 2024Shall We Team Up: Exploring Spontaneous Cooperation of Competing LLM Agentscited by 48
  13. 2023Theory of Mind in Large Language Models: Examining Performance of 11 State-of-the-Art models vs. Children Aged 7-10 on Advanced Testscited by 46
  14. 2024RS-Agent: Automating Remote Sensing Tasks through Intelligent Agentcited by 46
  15. 2024AgentSense: Benchmarking Social Intelligence of Language Agents through Interactive Scenarioscited by 37
  16. 2025Language Models use Lookbacks to Track Beliefscited by 36
  17. 2023Creativity Support in the Age of Large Language Models: An Empirical Study Involving Emerging Writerscited by 32
  18. 2023Empowering Psychotherapy with Large Language Models: Cognitive Distortion Detection through Diagnosis of Thought Promptingcited by 31
  19. 2024Language Models Represent Beliefs of Self and Otherscited by 29
  20. 2024Not All LLM Reasoners Are Created Equalcited by 29
  21. 2025SDPO: Segment-Level Direct Preference Optimization for Social Agentscited by 29
  22. 2025Hypothesis-Driven Theory-of-Mind Reasoning for Large Language Modelscited by 28
  23. 2024LLM-GLOBE: A Benchmark Evaluating the Cultural Values Embedded in LLM Outputcited by 27
  24. 2024AMONGAGENTS: Evaluating Large Language Models in the Interactive Text-Based Social Deduction Gamecited by 26
  25. 2024InterIntent: Investigating Social Intelligence of LLMs via Intention Understanding in an Interactive Game Contextcited by 25
  26. 2025Large Language Models Miss the Multi-Agent Markcited by 25
  27. 2024Learning Syntax Without Planting Trees: Understanding Hierarchical Generalization in Transformerscited by 24
  28. 2025Theory of Mind in Large Language Models: Assessment and Enhancementcited by 24
  29. 2023Inductive reasoning in humans and large language modelscited by 23
  30. 2023The Neuro-Symbolic Inverse Planning Engine (NIPE): Modeling Probabilistic Social Inferences from Linguistic Inputscited by 23
  31. 2025AutoToM: Scaling Model-based Mental Inference via Automated Agent Modelingcited by 23
  32. 2025Re-evaluating Theory of Mind evaluation in large language modelscited by 23
  33. 2024Perceptions to Beliefs: Exploring Precursory Inferences for Theory of Mind in Large Language Modelscited by 22
  34. 2025DSGBench: A Diverse Strategic Game Benchmark for Evaluating LLM-based Agents in Complex Decision-Making Environmentscited by 22
  35. 2025Emergent Coordination in Multi-Agent Language Modelscited by 22
  36. 2024Advancing Social Intelligence in AI Agents: Technical Challenges and Open Questionscited by 21
  37. 2025Sentient Agent as a Judge: Evaluating Higher-Order Social Cognition in Large Language Modelscited by 21
  38. 2023Position: Key Claims in LLM Research Have a Long Tail of Footnotescited by 20

50 most cited of 161 All 161 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Probing the Robustness of Theory of Mind in Large Language Models
  2. 2026Reasoning Promotes Robustness in Theory of Mind Tasks
  3. 2024OpenToM: A Comprehensive Benchmark for Evaluating Theory-of-Mind Reasoning Capabilities of Large Language Models
  4. 2023HI-TOM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Models
  5. 2024Mind Your Theory: Theory of Mind Goes Deeper Than Reasoning
  6. 2023Evaluating Large Language Models in Theory of Mind Tasks
  7. 2025Sensitivity Meets Sparsity: The Impact of Extremely Sparse Parameter Patterns on Theory-of-Mind of Large Language Models
  8. 2026Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models