# Multimodal Large Language Models Meet Multimodal Emotion Recognition and Reasoning: A Survey

arXiv: https://arxiv.org/abs/2509.24322 · HTML: https://mlsanity.com/paper/2509.24322/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning.
- Builds on 46 of its 172 references that are in the map.
- First posted on arXiv 29 Sep 2025.

## Placed in

- LLM reasoning / [Audio-language model reasoning](https://mlsanity.com/lr/audio-language-model-reasoning/)
- Multimodal LLMs / vision-language models / [Multimodal emotion understanding](https://mlsanity.com/mllm/multimodal-emotion-understanding/)

## Builds on (most cited first, 30 of 46)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [VisualBERT: A Simple and Performant Baseline for Vision and Language](https://mlsanity.com/paper/1908.03557/) (2019)
- [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://mlsanity.com/paper/2406.12793/) (2024)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality](https://mlsanity.com/paper/2304.14178/) (2023)
- [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023)
- [VideoChat: Chat-Centric Video Understanding](https://mlsanity.com/paper/2305.06355/) (2023)
- [VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs](https://mlsanity.com/paper/2406.07476/) (2024)
- [Qwen2.5-Omni Technical Report](https://mlsanity.com/paper/2503.20215/) (2025)
- [Yi: Open Foundation Models by 01.AI](https://mlsanity.com/paper/2403.04652/) (2024)
- [Baichuan 2: Open Large-scale Language Models](https://mlsanity.com/paper/2309.10305/) (2023)
- [MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning](https://mlsanity.com/paper/2310.09478/) (2023)
- [Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models](https://mlsanity.com/paper/2311.07919/) (2023)
- [Qwen2-Audio Technical Report](https://mlsanity.com/paper/2407.10759/) (2024)
- [Long Context Transfer from Language to Vision](https://mlsanity.com/paper/2406.16852/) (2024)
- [InternLM2 Technical Report](https://mlsanity.com/paper/2403.17297/) (2024)
- [MoE-LLaVA: Mixture of Experts for Large Vision-Language Models](https://mlsanity.com/paper/2401.15947/) (2024)
- [R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Reinforcement Learning](https://mlsanity.com/paper/2503.05379/) (2025)
- [Audio-Reasoner: Improving Reasoning Capability in Large Audio Language Models](https://mlsanity.com/paper/2503.02318/) (2025)
- [HumanOmni: A Large Vision-Speech Language Model for Human-Centric Video Understanding](https://mlsanity.com/paper/2501.15111/) (2025)
- [HumanOmniV2: From Understanding to Omni-Modal Reasoning with Context](https://mlsanity.com/paper/2506.21277/) (2025)
- [A Comprehensive Survey on Multi-modal Conversational Emotion Recognition with Deep Learning](https://mlsanity.com/paper/2312.05735/) (2023)

## Built on by (most cited first, 19 of 19)

- [The Paradigm Shift: A Comprehensive Survey on Large Vision Language Models for Multimodal Fake News Detection](https://mlsanity.com/paper/2601.15316/) (2026)
- [Dual-branch Graph Domain Adaptation for Cross-scenario Multi-modal Emotion Recognition](https://mlsanity.com/paper/2603.26840/) (2026)
- [MTLLFM: Multimodal-Temporal Laughter Localization: UR-FUNNY-Temporal and SMILE-Temporal Benchmarks with an Adaptive Multimodal Fusion Model](https://mlsanity.com/paper/2605.25409/) (2026)
- [A Flow Model with Low-Rank Transformers for Incomplete Multimodal Survival Analysis](https://mlsanity.com/paper/2510.21829/) (2025)
- [Learning to Hear by Seeing: It's Time for Vision Language Models to Understand Artistic Emotion from Sight and Sound](https://mlsanity.com/paper/2511.12077/) (2025)
- [Learning What to Attend First: Modality-Importance-Guided Reasoning for Reliable Multimodal Emotion Understanding](https://mlsanity.com/paper/2512.02699/) (2025)
- [MER-R1: Multimodal Emotion Reasoning via Slow-Fast Thinking Synergy](https://mlsanity.com/paper/2606.27652/) (2026)
- [Multilingual Emotion Neurons in Large Audio-Language Models](https://mlsanity.com/paper/2608.08772/) (2026)
- [TimeGNN-Augmented Hybrid-Action MARL for Fine-Grained Task Partitioning and Energy-Aware Offloading in MEC](https://mlsanity.com/paper/2601.06191/) (2026)
- [E^2-LLM: Bridging Neural Signals and Interpretable Affective Analysis](https://mlsanity.com/paper/2601.07877/) (2026)
- [Layout-Guided Controllable Pathology Image Generation with In-Context Diffusion Transformers](https://mlsanity.com/paper/2603.13386/) (2026)
- [Relational graph-driven differential denoising and diffusion attention fusion for multimodal conversation emotion recognition](https://mlsanity.com/paper/2603.25752/) (2026)
- [C2F-Thinker: Coarse-to-Fine Reasoning with Hint-Guided Reinforcement Learning for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2604.00013/) (2026)
- [Disentangled Dual-Branch Graph Learning for Conversational Emotion Recognition](https://mlsanity.com/paper/2604.14204/) (2026)
- [MAF: Multimodal Adaptive Few-shot Prompting for Sentiment Analysis with MLLMs](https://mlsanity.com/paper/2606.15694/) (2026)
- [Do We Really Need Multimodal Emotion Language Models Larger Than 1B Parameters?](https://mlsanity.com/paper/2607.12787/) (2026)
- [AtmosERC: Modeling Dialogue-Level Affective Atmosphere for Emotion Recognition in Conversation](https://mlsanity.com/paper/2607.26726/) (2026)
- [C2MOE: Consistency and Complementarity-guided Mixture of Experts for Incomplete Multimodal Emotion Learning](https://mlsanity.com/paper/2608.04013/) (2026)
- [Emotion Experience, Expression, and Perception: Emotion Analysis on Multimodal Social Media Posts](https://mlsanity.com/paper/2609.18385/) (2026)

## Abstract (arXiv)

In recent years, large language models (LLMs) have driven major advances in language understanding, marking a significant step toward artificial general intelligence (AGI). With increasing demands for higher-level semantics and cross-modal fusion, multimodal large language models (MLLMs) have emerged, integrating diverse information sources (e.g., text, vision, and audio) to enhance modeling and reasoning in complex scenarios. In AI for Science, multimodal emotion recognition and reasoning has become a rapidly growing frontier. While LLMs and MLLMs have achieved notable progress in this area, the field still lacks a systematic review that consolidates recent developments. To address this gap, this paper provides a comprehensive survey of LLMs and MLLMs for emotion recognition and reasoning, covering model architectures, datasets, and performance benchmarks. We further highlight key challenges and outline future research directions, aiming to offer researchers both an authoritative reference and practical insights for advancing this domain. To the best of our knowledge, this paper is the first attempt to comprehensively survey the intersection of MLLMs with multimodal emotion recognition and reasoning. The summary of existing methods mentioned is in our Github: hrefhttps://github.com/yuntaoshou/Awesome-Emotion-Reasoninghttps://github.com/yuntaoshou/Awesome-Emotion-Reasoning.