# Can MLLMs Read the Room? A Multimodal Benchmark for Verifying Truthfulness in Multi-Party Social Interactions

arXiv: https://arxiv.org/abs/2510.27195 · HTML: https://mlsanity.com/paper/2510.27195/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 36 references that are in the map.
- First posted on arXiv 31 Oct 2025.

## Builds on (most cited first, 9 of 9)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Multimodal Chain-of-Thought Reasoning in Language Models](https://mlsanity.com/paper/2302.00923/) (2023)
- [Fake News Detection on Social Media: A Data Mining Perspective](https://mlsanity.com/paper/1708.01967/) (2017)
- [MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos](https://mlsanity.com/paper/1606.06259/) (2016)
- [Multi-attention Recurrent Network for Human Communication Comprehension](https://mlsanity.com/paper/1802.00923/) (2018)
- [Modeling Multimodal Social Interactions: New Challenges and Baselines with Densely Aligned Representations](https://mlsanity.com/paper/2403.02090/) (2024)
- [Towards Online Multi-Modal Social Interaction Understanding](https://mlsanity.com/paper/2503.19851/) (2025)
- [SocialGesture: Delving into Multi-person Gesture Understanding](https://mlsanity.com/paper/2504.02244/) (2025)

## Built on by (most cited first, 6 of 6)

- [Multi-speaker Attention Alignment for Multimodal Social Interaction](https://mlsanity.com/paper/2511.17952/) (2025)
- [Can MLLMs Read the Room? A Multimodal Benchmark for Assessing Deception in Multi-Party Social Interactions](https://mlsanity.com/paper/2511.16221/) (2025)
- [H2HMem: A Multimodal Memory Benchmark for Agents in Human-Human Interactions](https://mlsanity.com/paper/2606.09461/) (2026)
- [GroupToM-Bench: Benchmarking Group Theory of Mind and Nonlinear Social Emergence in MLLMs](https://mlsanity.com/paper/2606.04184/) (2026)
- [GesBench: When and Where Do Multimodal Language Models Perceive Social Meaning?](https://mlsanity.com/paper/2610.15129/) (2026)
- [What Makes a Social Agent's Interactions Effective? Learning from Verifiable Outcomes in Paired Worlds](https://mlsanity.com/paper/2610.20195/) (2026)

## Abstract (arXiv)

As AI systems become increasingly integrated into human lives, endowing them with robust social intelligence has emerged as a critical frontier. A key aspect of this intelligence is discerning truth from deception, a ubiquitous element of human interaction that is conveyed through a complex interplay of verbal language and non-verbal visual cues. However, automatic deception detection in dynamic, multi-party conversations remains a significant challenge. The recent rise of powerful Multimodal Large Language Models (MLLMs), with their impressive abilities in visual and textual understanding, makes them natural candidates for this task. Consequently, their capabilities in this crucial domain are mostly unquantified. To address this gap, we introduce a new task, Multimodal Interactive Veracity Assessment (MIVA), and present a novel multimodal dataset derived from the social deduction game Werewolf. This dataset provides synchronized video, text, with verifiable ground-truth labels for every statement. We establish a comprehensive benchmark evaluating state-of-the-art MLLMs, revealing a significant performance gap: even powerful models like GPT-4o struggle to distinguish truth from falsehood reliably. Our analysis of failure modes indicates that these models fail to ground language in visual social cues effectively and may be overly conservative in their alignment, highlighting the urgent need for novel approaches to building more perceptive and trustworthy AI systems.