EmoMM: Benchmarking and Steering MLLM for Multimodal Emotion Recognition under Conflict and Missingness
Not on the map. Why?
Abstract (arXiv)
Multimodal Emotion Recognition (MER) is critical for interpreting real-world interactions. While Multimodal Large Language Models (MLLM) have shown promise in MER, their internal decision-making mechanisms under modality conflict and missingness remain largely underexplored. In this paper, to systematically investigate these behaviors, we introduce EmoMM, a comprehensive benchmark featuring modality-aligned, conflict, and missing subsets. Through extensive evaluation, we uncover a Video Contribution Collapse (VCC) phenomenon, where MLLM marginalize video evidence due to high token redundancy and modality preferences. To address this, we propose Conflict-aware Head-level Attention Steering (CHASE), a lightweight mechanism that detects modality conflicts and performs inference-time attention steering, effectively mitigating decision bias without retraining the backbone. Experimental results demonstrate that CHASE consistently improves performance across various settings, significantly enhancing the reliability of MLLM in complex affective scenarios.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 16 of its 33 references that are in the map. First posted on arXiv 1 May 2026.
Builds on
16 of 33 references.
References · 16
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
- 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
- 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,004
- 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
- 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 637
- 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 636
- 2017Tensor Fusion Network for Multimodal Sentiment Analysiscited by 429
- 2023Multimodal Deep Learningcited by 257
- 2016MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videoscited by 193
- 2025VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interactioncited by 191
- 2024VITA: Towards Open-Source Interactive Omni Multimodal LLMcited by 172
- 2024Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuningcited by 117
4 more
- 2024video-SALMONN: Speech-Enhanced Audio-Visual Large Language Modelscited by 101
- 2022Make Acoustic and Visual Cues Matter: CH-SIMS v2.0 Dataset and AV-Mixup Consistent Modulecited by 43
- 2025Benchmarking and Bridging Emotion Conflicts for Multimodal Emotion Reasoningcited by 12
- 2024Leveraging Retrieval Augment Approach for Multimodal Emotion Recognition Under Missing Modalitiescited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Pioneering Multimodal Emotion Recognition in the Era of Large Models: From Closed Sets to Open Vocabularies
- 2024EmoLLM: Multimodal Emotional Understanding Meets Large Language Models
- 2025Beyond Emotion Recognition: A Multi-Turn Multimodal Emotion Understanding and Reasoning Benchmark
- 2026EmoTrans: A Benchmark for Understanding, Reasoning, and Predicting Emotion Transitions in Multimodal LLMs
- 2025AffectGPT: A New Dataset, Model, and Benchmark for Emotion Understanding with Multimodal Large Language Models
- 2025Multimodal Large Language Models Meet Multimodal Emotion Recognition and Reasoning: A Survey
- 2025EmoBench-M: Benchmarking Emotional Intelligence for Multimodal Large Language Models
- 2026Emotion-LLaMAv2 and MMEVerse: A New Framework and Benchmark for Multimodal Emotion Understanding