# MMAFFBen: A Multilingual and Multimodal Affective Analysis Benchmark for Evaluating LLMs and VLMs

arXiv: https://arxiv.org/abs/2505.24423 · HTML: https://mlsanity.com/paper/2505.24423/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal emotion understanding, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 13 of its 54 references that are in the map.
- First posted on arXiv 30 May 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Multimodal emotion understanding](https://mlsanity.com/mllm/multimodal-emotion-understanding/)

## Builds on (most cited first, 13 of 13)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites](https://mlsanity.com/paper/2404.16821/) (2024)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performance](https://mlsanity.com/paper/2410.16261/) (2024)
- [Context Based Emotion Recognition using EMOTIC Dataset](https://mlsanity.com/paper/2003.13401/) (2020)
- [XED: A Multilingual Dataset for Sentiment Analysis and Emotion Detection](https://mlsanity.com/paper/2011.01612/) (2020)
- [Omni-Emotion: Extending Video MLLM with Detailed Face and Audio Modeling for Multimodal Emotion Analysis](https://mlsanity.com/paper/2501.09502/) (2025)
- [Emotion Detection and Analysis on Social Media](https://mlsanity.com/paper/1901.08458/) (2019)
- [Polarity and Intensity: the Two Aspects of Sentiment Analysis](https://mlsanity.com/paper/1807.01466/) (2018)

## Built on by (most cited first, 4 of 4)

- [Cross-Lingual Sentiment Misalignment: Auditing Multilingual Language Models for Inversion Risk, Dialectal Representation, and Affective Stability](https://mlsanity.com/paper/2602.17469/) (2026)
- [Do Emotions Influence Moral Judgment in Large Language Models?](https://mlsanity.com/paper/2604.19125/) (2026)
- [Do Vision Language Models Understand Human Engagement in Games?](https://mlsanity.com/paper/2603.18480/) (2026)
- [Measuring the Unspoken: A Disentanglement Model and Benchmark for Psychological Analysis in the Wild](https://mlsanity.com/paper/2512.04728/) (2025)

## Abstract (arXiv)

Large language models and vision-language models (which we jointly call LMs) have transformed NLP and CV, demonstrating remarkable potential across various fields. However, their capabilities in affective analysis (i.e. sentiment analysis and emotion detection) remain underexplored. This gap is largely due to the absence of comprehensive evaluation benchmarks, and the inherent complexity of affective analysis tasks. In this paper, we introduce MMAFFBen, the first extensive open-source benchmark for multilingual multimodal affective analysis. MMAFFBen encompasses text, image, and video modalities across 35 languages, covering four key affective analysis tasks: sentiment polarity, sentiment intensity, emotion classification, and emotion intensity. Moreover, we construct the MMAFFIn dataset for fine-tuning LMs on affective analysis tasks, and further develop MMAFFLM-3b and MMAFFLM-7b based on it. We evaluate various representative LMs, including GPT-4o-mini, providing a systematic comparison of their affective understanding capabilities. This project is available at https://github.com/lzw108/MMAFFBen.