# Building a Large Scale Dataset for Image Emotion Recognition: The Fine Print and The Benchmark

arXiv: https://arxiv.org/abs/1605.02677 · HTML: https://mlsanity.com/paper/1605.02677/

## Key facts

- Cited by 71 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 May 2016.

## Built on by (most cited first, 30 of 71)

- [Object Detection in Aerial Images: A Large-Scale Benchmark and Challenges](https://mlsanity.com/paper/2102.12219/) (2021)
- [ArtEmis: Affective Language for Visual Art](https://mlsanity.com/paper/2101.07396/) (2021)
- [HRS-Bench: Holistic, Reliable and Scalable Benchmark for Text-to-Image Models](https://mlsanity.com/paper/2304.05390/) (2023)
- [A Survey on Bias in Visual Datasets](https://mlsanity.com/paper/2107.07919/) (2021)
- [EmoSet: A Large-scale Visual Emotion Dataset with Rich Attributes](https://mlsanity.com/paper/2307.07961/) (2023)
- [Affective Image Content Analysis: Two Decades Review and New Perspectives](https://mlsanity.com/paper/2106.16125/) (2021)
- [RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressions](https://mlsanity.com/paper/2302.09466/) (2023)
- [EmoVIT: Revolutionizing Emotion Insights with Visual Instruction Tuning](https://mlsanity.com/paper/2404.16670/) (2024)
- [GPT-4V with Emotion: A Zero-shot Benchmark for Generalized Emotion Recognition](https://mlsanity.com/paper/2312.04293/) (2023)
- [MDAN: Multi-level Dependent Attention Network for Visual Emotion Analysis](https://mlsanity.com/paper/2203.13443/) (2022)
- [Stimuli-Aware Visual Emotion Analysis](https://mlsanity.com/paper/2109.01812/) (2021)
- [Affection: Learning Affective Explanations for Real-World Visual Data](https://mlsanity.com/paper/2210.01946/) (2022)
- [Evaluating Vision-Language Models for Emotion Recognition](https://mlsanity.com/paper/2502.05660/) (2025)
- [SOLVER: Scene-Object Interrelated Visual Emotion Reasoning Network](https://mlsanity.com/paper/2110.12334/) (2021)
- [UniEmo: Unifying Emotional Understanding and Generation with Learnable Expert Queries](https://mlsanity.com/paper/2507.23372/) (2025)
- [Make Me Happier: Evoking Emotions Through Image Diffusion Models](https://mlsanity.com/paper/2403.08255/) (2024)
- [Unlocking the Emotional World of Visual Media: An Overview of the Science, Research, and Impact of Understanding Emotion](https://mlsanity.com/paper/2307.13463/) (2023)
- [Affective Image Editing: Shaping Emotional Factors via Text Descriptions](https://mlsanity.com/paper/2505.18699/) (2025)
- [Towards Unbiased Visual Emotion Recognition via Causal Intervention](https://mlsanity.com/paper/2107.12096/) (2021)
- [EmoArt: A Multidimensional Dataset for Emotion-Aware Artistic Generation](https://mlsanity.com/paper/2506.03652/) (2025)
- [An AutoML-based Approach to Multimodal Image Sentiment Analysis](https://mlsanity.com/paper/2102.08092/) (2021)
- [Learning to Compose Diversified Prompts for Image Emotion Classification](https://mlsanity.com/paper/2201.10963/) (2022)
- [EmoVerse: A MLLMs-Driven Emotion Representation Dataset for Interpretable Visual Emotion Analysis](https://mlsanity.com/paper/2511.12554/) (2025)
- [Computational Emotion Analysis From Images: Recent Advances and Future Directions](https://mlsanity.com/paper/2103.10798/) (2021)
- [On the use of Vision-Language models for Visual Sentiment Analysis: a study on CLIP](https://mlsanity.com/paper/2310.12062/) (2023)
- [A Fair and Comprehensive Comparison of Multimodal Tweet Sentiment Analysis Methods](https://mlsanity.com/paper/2106.08829/) (2021)
- [Affective Visual Dialog: A Large-Scale Benchmark for Emotional Reasoning Based on Visually Grounded Conversations](https://mlsanity.com/paper/2308.16349/) (2023)
- [Vision-to-Music Generation: A Survey](https://mlsanity.com/paper/2503.21254/) (2025)
- [MGHFT: Multi-Granularity Hierarchical Fusion Transformer for Cross-Modal Sticker Emotion Recognition](https://mlsanity.com/paper/2507.18929/) (2025)
- [EmoCaliber: Advancing Reliable Visual Emotion Comprehension via Confidence Verbalization and Calibration](https://mlsanity.com/paper/2512.15528/) (2025)

## Abstract (arXiv)

Psychological research results have confirmed that people can have different emotional reactions to different visual stimuli. Several papers have been published on the problem of visual emotion analysis. In particular, attempts have been made to analyze and predict people's emotional reaction towards images. To this end, different kinds of hand-tuned features are proposed. The results reported on several carefully selected and labeled small image data sets have confirmed the promise of such features. While the recent successes of many computer vision related tasks are due to the adoption of Convolutional Neural Networks (CNNs), visual emotion analysis has not achieved the same level of success. This may be primarily due to the unavailability of confidently labeled and relatively large image data sets for visual emotion analysis. In this work, we introduce a new data set, which started from 3+ million weakly labeled images of different emotions and ended up 30 times as large as the current largest publicly available visual emotion data set. We hope that this data set encourages further research on visual emotion analysis. We also perform extensive benchmarking analyses on this large data set using the state of the art methods including CNNs.