# Multimodal LLMs See Sentiment

arXiv: https://arxiv.org/abs/2508.16873 · HTML: https://mlsanity.com/paper/2508.16873/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal emotion understanding, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 26 of its 54 references that are in the map.
- First posted on arXiv 23 Aug 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Multimodal emotion understanding](https://mlsanity.com/mllm/multimodal-emotion-understanding/)

## Builds on (most cited first, 26 of 26)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [DeepSeek-VL: Towards Real-World Vision-Language Understanding](https://mlsanity.com/paper/2403.05525/) (2024)
- [EVA: Exploring the Limits of Masked Visual Representation Learning at Scale](https://mlsanity.com/paper/2211.07636/) (2022)
- [DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding](https://mlsanity.com/paper/2412.10302/) (2024)
- [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://mlsanity.com/paper/2412.13663/) (2024)
- [Language Is Not All You Need: Aligning Perception with Language Models](https://mlsanity.com/paper/2302.14045/) (2023)
- [MM-LLMs: Recent Advances in MultiModal Large Language Models](https://mlsanity.com/paper/2401.13601/) (2024)
- [Sentiment Analysis in the Era of Large Language Models: A Reality Check](https://mlsanity.com/paper/2305.15005/) (2023)
- [Robust Image Sentiment Analysis Using Progressively Trained and Domain Transferred Deep Networks](https://mlsanity.com/paper/1509.06041/) (2015)
- [MLLM-CompBench: A Comparative Reasoning Benchmark for Multimodal LLMs](https://mlsanity.com/paper/2407.16837/) (2024)
- [Unveiling the Ignorance of MLLMs: Seeing Clearly, Answering Incorrectly](https://mlsanity.com/paper/2406.10638/) (2024)
- [DeepSentiBank: Visual Sentiment Concept Classification with Deep Convolutional Neural Networks](https://mlsanity.com/paper/1410.8586/) (2014)
- [Evaluating MLLMs with Multimodal Multi-image Reasoning Benchmark](https://mlsanity.com/paper/2506.04280/) (2025)
- [From Pixels to Sentiment: Fine-tuning CNNs for Visual Sentiment Prediction](https://mlsanity.com/paper/1604.03489/) (2016)
- [Why Only Text: Empowering Vision-and-Language Navigation with Multi-modal Prompts](https://mlsanity.com/paper/2406.02208/) (2024)
- [Exploring Cognitive and Aesthetic Causality for Multimodal Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2504.15848/) (2025)
- [Survey on Visual Sentiment Analysis](https://mlsanity.com/paper/2004.11639/) (2020)
- [On the use of Vision-Language models for Visual Sentiment Analysis: a study on CLIP](https://mlsanity.com/paper/2310.12062/) (2023)
- [Towards Explainable Fusion and Balanced Learning in Multimodal Sentiment Analysis](https://mlsanity.com/paper/2504.12151/) (2025)

## Built on by (most cited first, 2 of 2)

- [Stable Behavior, Limited Variation: Persona Validity in LLM Agents for Urban Sentiment Perception](https://mlsanity.com/paper/2604.28048/) (2026)
- [Grandes Modelos de Linguagem Multimodais (MLLMs): Da Teoria `a Pr'atica](https://mlsanity.com/paper/2602.12302/) (2026)

## Abstract (arXiv)

Understanding how visual content conveys sentiment is increasingly important in a digital landscape dominated by imagery. However, sentiment perception depends on complex scene-level semantics, making this a challenging task for computational models. This paper examines how Multimodal Large Language Models (MLLMs) perform sentiment analysis in images through a systematic, evaluation-driven study encompassing three perspectives: (i) direct sentiment classification from images using MLLMs; (ii) sentiment analysis on MLLM-generated descriptions using pre-trained LLMs; and (iii) fine-tuning these LLMs on sentiment-labeled descriptions to assess performance and generalization. Experiments on a recent benchmark show that a two-stage MLLM description-mediated pipeline can substantially improve prediction accuracy under several evaluation settings, particularly when the LLM component is fine-tuned. Across different agreement thresholds and sentiment granularities, the strongest configurations of this pipeline outperform lexicon-, CNN-, and Transformer-based baselines in our benchmark by up to 30.9%, 64.8%, and 42.4%, respectively. In cross-dataset evaluation, the proposed pipeline - without training or fine-tuning on the target dataset - still surpasses the best in-domain baseline by over 8%. Overall, the study provides a comprehensive assessment of MLLM description-mediated sentiment analysis, clarifying the conditions under which it is effective, the scenarios in which it fails, and its comparison with traditional vision-based approaches, while also providing a reproducible benchmark resource for future research.