# MERBench: A Unified Evaluation Benchmark for Multimodal Emotion Recognition

arXiv: https://arxiv.org/abs/2401.03429 · HTML: https://mlsanity.com/paper/2401.03429/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jan 2024.

## Built on by (most cited first, 27 of 27)

- [Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems](https://mlsanity.com/paper/2504.01990/) (2025)
- [EmoBench-M: Benchmarking Emotional Intelligence for Multimodal Large Language Models](https://mlsanity.com/paper/2502.04424/) (2025)
- [Affective Computing in the Era of Large Language Models: A Survey from the NLP Perspective](https://mlsanity.com/paper/2408.04638/) (2024)
- [Recent Advances in Multimodal Affective Computing: An NLP Perspective](https://mlsanity.com/paper/2409.07388/) (2024)
- [MEMO-Bench: A Multiple Benchmark for Text-to-Image and Multimodal Large Language Models on Human Emotion Analysis](https://mlsanity.com/paper/2411.11235/) (2024)
- [ComBodied Agents: a New Paradigm of Human-Centric Agentic AI](https://mlsanity.com/paper/2608.10915/) (2026)
- [A Principled Framework for Multi-View Contrastive Learning](https://mlsanity.com/paper/2507.06979/) (2025)
- [CyIN: Cyclic Informative Latent Space for Bridging Complete and Incomplete Multimodal Learning](https://mlsanity.com/paper/2602.04920/) (2026)
- [DeepMLF: Multimodal language model with learnable tokens for deep fusion in sentiment analysis](https://mlsanity.com/paper/2504.11082/) (2025)
- [Evaluation of data inconsistency for multi-modal sentiment analysis](https://mlsanity.com/paper/2406.03004/) (2024)
- [Enhancing Multimodal Sentiment Analysis for Missing Modality through Self-Distillation and Unified Modality Cross-Attention](https://mlsanity.com/paper/2410.15029/) (2024)
- [Multimodal Fusion with Pre-Trained Model Features in Affective Behaviour Analysis In-the-wild](https://mlsanity.com/paper/2403.15044/) (2024)
- [Early Joint Learning of Emotion Information Makes MultiModal Model Understand You Better](https://mlsanity.com/paper/2409.18971/) (2024)
- [Explainable Multimodal Depression Recognition in Clinical Interviews via PHQ-Aligned Symptom Summarization](https://mlsanity.com/paper/2501.16106/) (2025)
- [Who is a Better Player: LLM against LLM](https://mlsanity.com/paper/2508.04720/) (2025)
- [DashFusion: Dual-stream Alignment with Hierarchical Bottleneck Fusion for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2512.05515/) (2025)
- [XEmoGPT: An Explainable Multimodal Emotion Recognition Framework with Cue-Level Perception and Reasoning](https://mlsanity.com/paper/2602.05496/) (2026)
- [Memory Bear AI Memory Science Engine for Multimodal Affective Intelligence: A Technical Report](https://mlsanity.com/paper/2603.22306/) (2026)
- [EmoScene: A Dual-space Dataset for Controllable Affective Image Generation](https://mlsanity.com/paper/2604.00933/) (2026)
- [Retrieval-Augmented Personalization with Foundation Models for Wearable Stress Detection](https://mlsanity.com/paper/2606.24985/) (2026)
- [Fitting Different Interactive Information: Joint Classification of Emotion and Intention](https://mlsanity.com/paper/2501.06215/) (2025)
- [OmniMER: Auxiliary-Enhanced LLM Adaptation for Indonesian Multimodal Emotion Recognition](https://mlsanity.com/paper/2512.19379/) (2025)
- [AMB-DSGDN: Adaptive Modality-Balanced Dynamic Semantic Graph Differential Network for Multimodal Emotion Recognition](https://mlsanity.com/paper/2603.10043/) (2026)
- [Facial-Expression-Aware Prompting for Empathetic LLM Tutoring](https://mlsanity.com/paper/2604.15336/) (2026)
- [Ordering Matters: Rank-Aware Selective Fusion for Blended Emotion Recognition](https://mlsanity.com/paper/2605.21417/) (2026)
- [EmotionDialogCN: A Spontaneous Multimodal Dataset for Mandarin Emotional Dialogue](https://mlsanity.com/paper/2608.20905/) (2026)
- [LongEmo: Towards Emotion Understanding and Reasoning in Long Videos](https://mlsanity.com/paper/2609.40079/) (2026)

## Abstract (arXiv)

Multimodal emotion recognition plays a crucial role in enhancing user experience in human-computer interaction. Over the past few decades, researchers have proposed a series of algorithms and achieved impressive progress. Although each method shows its superior performance, different methods lack a fair comparison due to inconsistencies in feature extractors, evaluation manners, and experimental settings. These inconsistencies severely hinder the development of this field. Therefore, we build MERBench, a unified evaluation benchmark for multimodal emotion recognition. We aim to reveal the contribution of some important techniques employed in previous works, such as feature selection, multimodal fusion, robustness analysis, fine-tuning, pre-training, etc. We hope this benchmark can provide clear and comprehensive guidance for follow-up researchers. Based on the evaluation results of MERBench, we further point out some promising research directions. Additionally, we introduce a new emotion dataset MER2023, focusing on the Chinese language environment. This dataset can serve as a benchmark dataset for research on multi-label learning, noise robustness, and semi-supervised learning. We encourage the follow-up researchers to evaluate their algorithms under the same experimental setup as MERBench for fair comparisons. Our code is available at: https://github.com/zeroQiaoba/MERTools.