mlsanityData to 7 Oct 2026

NegotiationToM: A Benchmark for Stress-testing Machine Theory of Mind on Negotiation Surrounding

arXiv 2404.13627 ↗First posted 21 Apr 2024cs.CL cs.AIcited by 49

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have sparked substantial interest and debate concerning their potential emergence of Theory of Mind (ToM) ability. Theory of mind evaluations currently focuses on testing models using machine-generated data or game settings prone to shortcuts and spurious correlations, which lacks evaluation of machine ToM ability in real-world human interaction scenarios. This poses a pressing demand to develop new real-world scenario benchmarks. We introduce NegotiationToM, a new benchmark designed to stress-test machine ToM in real-world negotiation surrounding covered multi-dimensional mental states (i.e., desires, beliefs, and intentions). Our benchmark builds upon the Belief-Desire-Intention (BDI) agent modeling theory and conducts the necessary empirical experiments to evaluate large language models. Our findings demonstrate that NegotiationToM is challenging for state-of-the-art LLMs, as they consistently perform significantly worse than humans, even when employing the chain-of-thought (CoT) method.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Builds on 38 of its 104 references that are in the map. First posted on arXiv 21 Apr 2024.

Builds on

38 of 104 references.

References · 38

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  5. 2017Deep reinforcement learning from human preferencescited by 3,846
  6. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  7. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  8. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 324
  9. 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 288
  10. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
  11. 2023Mathematical Capabilities of ChatGPTcited by 263
  12. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
26 more
  1. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  2. 2017Deal or No Deal? End-to-End Learning for Negotiation Dialoguescited by 202
  3. 2022Leveraging Large Language Models for Multiple Choice Question Answeringcited by 199
  4. 2023Time Travel in LLMs: Tracing Data Contamination in Large Language Modelscited by 175
  5. 2023Understanding Social Reasoning in Language Models with Language Modelscited by 166
  6. 2023FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactionscited by 162
  7. 2023Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Modelscited by 161
  8. 2018Decoupling Strategy and Generation in Negotiation Dialoguescited by 156
  9. 2023Task Contamination: Language Models May Not Be Few-Shot Anymorecited by 98
  10. 2023Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Trackercited by 95
  11. 2022ChatGPT: The End of Online Exam Integrity?cited by 88
  12. 2021Template-free Prompt Tuning for Few-shot NERcited by 73
  13. 2018Evaluating Theory of Mind in Question Answeringcited by 64
  14. 2021CaSiNo: A Corpus of Campsite Negotiation Dialogues for Automatic Negotiation Systemscited by 60
  15. 2023ChatGPT Evaluation on Sentence Level Relations: A Focus on Temporal, Causal, and Discourse Relationscited by 39
  16. 2020Improving Dialog Systems for Negotiation with Personality Modelingcited by 37
  17. 2023PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Modelscited by 35
  18. 2022Let's Negotiate! A Survey of Negotiation Dialogue Systemscited by 31
  19. 2024CANDLE: Iterative Conceptualization and Instantiation Distillation from Large Language Models for Commonsense Reasoningcited by 28
  20. 2023StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understandingcited by 26
  21. 2023DiscoPrompt: Path Prediction Prompt Tuning for Implicit Discourse Relation Recognitioncited by 23
  22. 2022Mind the gap: Challenges of deep learning approaches to Theory of Mindcited by 21
  23. 2020Few-shot Learning for Multi-label Intent Detectioncited by 20
  24. 2018Machine Common Sense Concept Papercited by 17
  25. 2023Self-Consistent Narrative Prompts on Abductive Natural Language Inferencecited by 11
  26. 2024EventGround: Narrative Reasoning by Grounding to Eventuality-centric Knowledge Graphscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 25; 2026: 17.

49 papers.

Sort:

By area:

  1. 2024MuMA-ToM: Multi-modal Multi-Agent Theory of Mindcited by 51
  2. 2025Language Models use Lookbacks to Track Beliefscited by 36
  3. 2025Theory of Mind in Large Language Models: Assessment and Enhancementcited by 24
  4. 2024Evaluating and Enhancing LLMs Agent based on Theory of Mind in Guandan: A Multi-Player Cooperative Game under Imperfect Informationcited by 20
  5. 2024ECon: On the Detection and Resolution of Evidence Conflictscited by 18
  6. 2025PersuasiveToM: A Benchmark for Evaluating Machine Theory of Mind in Persuasive Dialoguescited by 16
  7. 2025SoMi-ToM: Evaluating Multi-Perspective Theory of Mind in Embodied Social Interactionscited by 16
  8. 2025Rethinking Theory of Mind Benchmarks for LLMs: Towards A User-Centered Perspectivecited by 14
  9. 2024EgoSocialArena: Benchmarking the Social Intelligence of Large Language Models from a First-person Perspectivecited by 10
  10. 2025EcomScriptBench: A Multi-task Benchmark for E-commerce Script Planning via Step-wise Intention-Driven Product Associationcited by 10
  11. 2025Do Large Language Models Have a Planning Theory of Mind? Evidence from MindGames: a Multi-Step Persuasion Taskcited by 9
  12. 2025The Essence of Contextual Understanding in Theory of Mind: A Study on Question Answering with Story Characterscited by 8
37 more
  1. 2025Persuasion Should be Double-Blind: A Multi-Domain Dialogue Dataset With Faithfulness Based on Causal Theory of Mindcited by 8
  2. 2025LLM-Hanabi: Evaluating Multi-Agent Gameplays with Theory-of-Mind and Rationale Inference in Imperfect Information Collaboration Gamecited by 8
  3. 2025HARBOR: Exploring Persona Dynamics in Multi-Agent Competitioncited by 6
  4. 2024Constrained Reasoning Chains for Enhancing Theory-of-Mind in Large Language Modelscited by 5
  5. 2025ToM-SSI: Evaluating Theory of Mind in Situated Social Interactionscited by 5
  6. 2025TactfulToM: Do LLMs Have the Theory of Mind Ability to Understand White Lies?cited by 5
  7. 2025Evaluating Multi-Turn Bargain Skills in LLM-Based Seller Agentcited by 4
  8. 2024On the Role of Entity and Event Level Conceptualization in Generalizable Reasoning: A Survey of Tasks, Methods, Applications, and Future Directionscited by 3
  9. 2025Enhancing Conversational Agents with Theory of Mind: Aligning Beliefs, Desires, and Intentions for Human-Like Interactioncited by 3
  10. 2025Exploring Consciousness in LLMs: A Systematic Survey of Theories, Implementations, and Frontier Riskscited by 3
  11. 2025XToM: Exploring the Multilingual Theory of Mind for Large Language Modelscited by 2
  12. 2025Structuring the Unstructured: A Systematic Review of Text-to-Structure Generation for Agentic AI with a Universal Evaluation Frameworkcited by 2
  13. 2025HugAgent: A Human Simulation Benchmark for Individual-Level Reasoningcited by 2
  14. 2026Think Thrice Before You Speak: Dual knowledge-enhanced Theory-of-Mind Reasoning for Persuasive Agentscited by 2
  15. 2025PolicyEvol-Agent: Evolving Policy via Environment Perception and Self-Awareness with Theory of Mindcited by 1
  16. 2026CogToM: A Comprehensive Theory of Mind Benchmark inspired by Human Cognition for Large Language Modelscited by 1
  17. 2026To Think or Not To Think, That is The Question for Large Reasoning Models in Theory of Mind Taskscited by 1
  18. 2026Beyond Words: Evaluating and Bridging Epistemic Divergence in User-Agent Interaction via Theory of Mindcited by 1
  19. 2026Large Language Models Persuade Without Planning Theory of Mindcited by 1
  20. 2026SAGE: A Quantitative Evaluation of Socialized Evolution in Agent Ecosystemscited by 1
  21. 2026GroupToM-Bench: Benchmarking Group Theory of Mind and Nonlinear Social Emergence in MLLMscited by 1
  22. 2024Persona Knowledge-Aligned Prompt Tuning Method for Online Debatecited by 0
  23. 2025textttDIAMONDs: A Dataset for mathbbDynamic mathbbInformation mathbbAnd mathbbMental modeling mathbbOf mathbbNumeric mathbbDiscussionscited by 0
  24. 2025Effects of Theory of Mind and Prosocial Beliefs on Steering Human-Aligned Behaviors of LLMs in Ultimatum Gamescited by 0
  25. 2025Circuit Distillationcited by 0
  26. 2025Spot The Ball: A Benchmark for Visual Social Inferencecited by 0
  27. 2025S3IT: A Benchmark for Spatially Situated Social Intelligence Testcited by 0
  28. 2026Traces of Social Competence in Large Language Modelscited by 0
  29. 2026CoMMET: A Psychologically Grounded Benchmark for Evaluating Theory of Mind in Multimodal LLMscited by 0
  30. 2026CoSToM:Causal-oriented Steering for Intrinsic Theory-of-Mind Alignment in Large Language Modelscited by 0
  31. 2026Preference Estimation via Opponent Modeling in Multi-Agent Negotiationcited by 0
  32. 2026Does Theory of Mind Improvement Really Benefit Human-AI Interactions? Empirical Findings from Interactive Evaluationscited by 0
  33. 2026PrefBench: Evaluating Zero-Shot LLM Agents in Hidden-Preference Personalized Pricing Negotiationscited by 0
  34. 2026Theory of Mind and Persuasion Beyond Conversation: Assessing the Capacity of LLMs to Induce Belief States via Planning and Actioncited by 0
  35. 2026Towards High-Level Semantic Intelligencecited by 0
  36. 2026Can LLM Agents Price Competitively? A Dynamic Multi-Attribute Auction Benchmark for Agentic Commercecited by 0
  37. 2026Belief Without Behavior: Measuring the Translation of Theory of Mind into Coordinated Social Action in Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024The Good, the Bad, and the Hulk-like GPT: Analyzing Emotional Decisions of Large Language Models in Cooperation and Bargaining Games
  2. 2024SimpleToM: Exposing the Gap between Explicit ToM Inference and Implicit ToM Application in LLMs
  3. 2024Simulating Human Strategic Behavior: Comparing Single and Multi-agent LLMs
  4. 2026MIND: Multi-agent inference for negotiation dialogue in travel planning
  5. 2024OpenToM: A Comprehensive Benchmark for Evaluating Theory-of-Mind Reasoning Capabilities of Large Language Models
  6. 2025Humanizing LLMs: A Survey of Psychological Measurements with Tools, Datasets, and Human-Agent Applications
  7. 2026PieArena: Ranking and Profiling Language Agents in Realistic Negotiation Scenarios
  8. 2025The Decrypto Benchmark for Multi-Agent Reasoning and Theory of Mind