# MultiEmo-Bench: Multi-label Visual Emotion Analysis for Multi-modal Large Language Models

arXiv: https://arxiv.org/abs/2605.14635 · HTML: https://mlsanity.com/paper/2605.14635/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal emotion understanding, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 25 of its 36 references that are in the map.
- First posted on arXiv 14 May 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Multimodal emotion understanding](https://mlsanity.com/mllm/multimodal-emotion-understanding/)

## Builds on (most cited first, 25 of 25)

- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [ShareGPT4V: Improving Large Multi-Modal Models with Better Captions](https://mlsanity.com/paper/2311.12793/) (2023)
- [Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning](https://mlsanity.com/paper/2406.11161/) (2024)
- [ArtEmis: Affective Language for Visual Art](https://mlsanity.com/paper/2101.07396/) (2021)
- [EmoSet: A Large-scale Visual Emotion Dataset with Rich Attributes](https://mlsanity.com/paper/2307.07961/) (2023)
- [Building a Large Scale Dataset for Image Emotion Recognition: The Fine Print and The Benchmark](https://mlsanity.com/paper/1605.02677/) (2016)
- [AesBench: An Expert Benchmark for Multimodal Large Language Models on Image Aesthetics Perception](https://mlsanity.com/paper/2401.08276/) (2024)
- [AesExpert: Towards Multi-modality Foundation Model for Image Aesthetics Perception](https://mlsanity.com/paper/2404.09624/) (2024)
- [Affective Image Content Analysis: Two Decades Review and New Perspectives](https://mlsanity.com/paper/2106.16125/) (2021)
- [EmoVIT: Revolutionizing Emotion Insights with Visual Instruction Tuning](https://mlsanity.com/paper/2404.16670/) (2024)
- [It is Okay to Not Be Okay: Overcoming Emotional Bias in Affective Image Captioning by Contrastive Data Collection](https://mlsanity.com/paper/2204.07660/) (2022)
- [EmotiCrafter: Text-to-Emotional-Image Generation based on Valence-Arousal Model](https://mlsanity.com/paper/2501.05710/) (2025)
- [FindingEmo: An Image Dataset for Emotion Recognition in the Wild](https://mlsanity.com/paper/2402.01355/) (2024)
- [Evaluating Vision-Language Models for Emotion Recognition](https://mlsanity.com/paper/2502.05660/) (2025)
- [HumanAesExpert: Advancing a Multi-Modality Foundation Model for Human Image Aesthetic Assessment](https://mlsanity.com/paper/2503.23907/) (2025)
- [OV-MER: Towards Open-Vocabulary Multimodal Emotion Recognition](https://mlsanity.com/paper/2410.01495/) (2024)
- [EEmo-Bench: A Benchmark for Multi-modal Large Language Models on Image Evoked Emotion Assessment](https://mlsanity.com/paper/2504.16405/) (2025)
- [UniQA: Unified Vision-Language Pre-training for Image Quality and Aesthetic Assessment](https://mlsanity.com/paper/2406.01069/) (2024)
- [AffectGPT: Dataset and Framework for Explainable Multimodal Emotion Recognition](https://mlsanity.com/paper/2407.07653/) (2024)
- [EmoArt: A Multidimensional Dataset for Emotion-Aware Artistic Generation](https://mlsanity.com/paper/2506.03652/) (2025)
- [EmoVerse: A MLLMs-Driven Emotion Representation Dataset for Interpretable Visual Emotion Analysis](https://mlsanity.com/paper/2511.12554/) (2025)
- [Socratis: Are large multimodal models emotionally aware?](https://mlsanity.com/paper/2308.16741/) (2023)
- [Why We Feel: Breaking Boundaries in Emotional Reasoning with Multimodal Large Language Models](https://mlsanity.com/paper/2504.07521/) (2025)

## Abstract (arXiv)

This paper introduces a multi-label visual emotion analysis benchmark dataset for comprehensively evaluating the ability of multimodal large language models (MLLMs) to predict the emotions evoked by images. Recent user studies report an unintuitive finding: humans may prefer the predictions of MLLMs over the labels in existing datasets. We argue that this phenomenon stems from the suboptimal annotation scheme used in existing datasets, where each annotator is shown a single candidate emotion for each image and judges whether it is evoked or not. This approach is clearly limited because a single image can evoke multiple emotions with varying intensities. As a result, evaluations based on these datasets may underestimate the capabilities of MLLMs, yet an appropriate benchmark for evaluating such models remains lacking. To address this issue, we introduce a new multi-label benchmark dataset for visual emotion analysis toward MLLMs evaluation. We hire 20 annotators per image and ask them to select all emotions they feel from an image. Then, we aggregate the votes across all annotators, providing a more reliable and representative dataset labeled with a distribution of emotions. The resulting dataset contains 10,344 images with 236,998 valid votes across eight emotions. Based on this benchmark dataset, we evaluate several recent models, including Qwen3-VL, OpenAI's GPT, Gemini, and Claude. We assess model performance on both dominant emotion prediction and emotion distribution prediction. Our results demonstrate the progress achieved by recent MLLMs while also indicating that substantial room for improvement remains. Furthermore, our experiments with LLM-as-a-judge show that the method does not consistently improve MLLMs' performance, indicating its limitations for the subjective task of visual emotion analysis.