# Can MLLMs Read the Room? A Multimodal Benchmark for Assessing Deception in Multi-Party Social Interactions

arXiv: https://arxiv.org/abs/2511.16221 · HTML: https://mlsanity.com/paper/2511.16221/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 21 of its 50 references that are in the map.
- First posted on arXiv 20 Nov 2025.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)
- Multimodal LLMs / vision-language models / [MLLM training and visual instruction tuning](https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/)

## Builds on (most cited first, 21 of 21)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos](https://mlsanity.com/paper/1606.06259/) (2016)
- [Benchmarking Robustness of 3D Object Detection to Common Corruptions in Autonomous Driving](https://mlsanity.com/paper/2303.11040/) (2023)
- [Multi-attention Recurrent Network for Human Communication Comprehension](https://mlsanity.com/paper/1802.00923/) (2018)
- [Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency](https://mlsanity.com/paper/2501.04931/) (2025)
- [ViewFool: Evaluating the Robustness of Visual Recognition to Adversarial Viewpoints](https://mlsanity.com/paper/2210.03895/) (2022)
- [Modeling Multimodal Social Interactions: New Challenges and Baselines with Densely Aligned Representations](https://mlsanity.com/paper/2403.02090/) (2024)
- [DIFFender: Diffusion-Based Adversarial Defense against Patch Attacks](https://mlsanity.com/paper/2306.09124/) (2023)
- [Towards Online Multi-Modal Social Interaction Understanding](https://mlsanity.com/paper/2503.19851/) (2025)
- [SocialGesture: Delving into Multi-person Gesture Understanding](https://mlsanity.com/paper/2504.02244/) (2025)
- [From reactive to cognitive: brain-inspired spatial intelligence for embodied agents](https://mlsanity.com/paper/2508.17198/) (2025)
- [Can MLLMs Read the Room? A Multimodal Benchmark for Verifying Truthfulness in Multi-Party Social Interactions](https://mlsanity.com/paper/2510.27195/) (2025)
- [Real-world Adversarial Defense against Patch Attacks based on Diffusion Model](https://mlsanity.com/paper/2409.09406/) (2024)
- [OODFace: Benchmarking Robustness of Face Recognition under Common Corruptions and Appearance Variations](https://mlsanity.com/paper/2412.02479/) (2024)

## Built on by (most cited first, 6 of 6)

- [Vinci2: Providing Proactive Assistance in Continuous Egocentric Videos](https://mlsanity.com/paper/2607.11523/) (2026)
- [SocialDirector: Training-Free Social Interaction Control for Multi-Person Video Generation](https://mlsanity.com/paper/2605.10079/) (2026)
- [GRASP: Learning to Ground Social Reasoning in Multi-Person Non-Verbal Interactions](https://mlsanity.com/paper/2605.15764/) (2026)
- [Perception or Prejudice: Can MLLMs Go Beyond First Impressions of Personality?](https://mlsanity.com/paper/2605.22109/) (2026)
- [HelloWorld: Enabling Socially Interactive Characters in Video World Models](https://mlsanity.com/paper/2608.05070/) (2026)
- [What Makes a Social Agent's Interactions Effective? Learning from Verifiable Outcomes in Paired Worlds](https://mlsanity.com/paper/2610.20195/) (2026)

## Abstract (arXiv)

Despite their advanced reasoning capabilities, state-of-the-art Multimodal Large Language Models (MLLMs) still lack a core component of human intelligence: the ability to "read the room" and assess deception in complex social interactions. To rigorously evaluate this limitation, we introduce Multimodal Interactive Deception Assessment (MIDA), a new task and multimodal dataset comprising synchronized video and text with verifiable ground-truth labels for every statement. We benchmark 12 state-of-the-art open- and closed-source MLLMs and identify a substantial performance gap: even powerful models such as GPT-4o struggle to reliably distinguish truthful statements from deceptive ones. Our analysis shows that current models have difficulty grounding language in multimodal social cues and modeling what others know, believe, or intend. To address these limitations, we propose a Social Chain-of-Thought (SoCoT) reasoning pipeline and a Dynamic Social Epistemic Memory (DSEM) module. Experimental results show that structured epistemic memory consistently improves deception assessment, while multimodal social-cue reasoning remains a challenging and important direction for future research.