# Affective Image Content Analysis: Two Decades Review and New Perspectives

arXiv: https://arxiv.org/abs/2106.16125 · HTML: https://mlsanity.com/paper/2106.16125/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 197 references that are in the map.
- First posted on arXiv 30 Jun 2021.

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [Domain Generalization via Invariant Feature Representation](https://mlsanity.com/paper/1301.2115/) (2013)
- [Hypergraph Neural Networks](https://mlsanity.com/paper/1809.09401/) (2018)
- [Deep Facial Expression Recognition: A Survey](https://mlsanity.com/paper/1804.08348/) (2018)
- [Deep Learning for Sentiment Analysis : A Survey](https://mlsanity.com/paper/1801.07883/) (2018)
- [A Review of Single-Source Deep Unsupervised Visual Domain Adaptation](https://mlsanity.com/paper/2009.00155/) (2020)
- [Context Based Emotion Recognition using EMOTIC Dataset](https://mlsanity.com/paper/2003.13401/) (2020)
- [Building a Large Scale Dataset for Image Emotion Recognition: The Fine Print and The Benchmark](https://mlsanity.com/paper/1605.02677/) (2016)
- [Robust Image Sentiment Analysis Using Progressively Trained and Domain Transferred Deep Networks](https://mlsanity.com/paper/1509.06041/) (2015)
- [Contemplating Visual Emotions: Understanding and Overcoming Dataset Bias](https://mlsanity.com/paper/1808.02212/) (2018)
- [Learning Multi-level Deep Representations for Image Emotion Classification](https://mlsanity.com/paper/1611.07145/) (2016)
- [An End-to-End Visual-Audio Attention Network for Emotion Recognition in User-Generated Videos](https://mlsanity.com/paper/2003.00832/) (2020)
- [DeepSentiBank: Visual Sentiment Concept Classification with Deep Convolutional Neural Networks](https://mlsanity.com/paper/1410.8586/) (2014)
- [MADAN: Multi-source Adversarial Domain Aggregation Network for Domain Adaptation](https://mlsanity.com/paper/2003.00820/) (2020)
- [EmotiW 2018: Audio-Video, Student Engagement and Group-Level Affect Prediction](https://mlsanity.com/paper/1808.07773/) (2018)
- [Affective Computing for Large-Scale Heterogeneous Multimedia Data: A Survey](https://mlsanity.com/paper/1911.05609/) (2019)
- [What Twitter Profile and Posted Images Reveal About Depression and Anxiety](https://mlsanity.com/paper/1904.02670/) (2019)
- [PDANet: Polarity-consistent Deep Attention Network for Fine-grained Visual Emotion Regression](https://mlsanity.com/paper/1909.05693/) (2019)
- [Multi-source Domain Adaptation for Visual Sentiment Classification](https://mlsanity.com/paper/2001.03886/) (2020)
- [From Pixels to Sentiment: Fine-tuning CNNs for Visual Sentiment Prediction](https://mlsanity.com/paper/1604.03489/) (2016)
- [Sentiment Analysis from Images of Natural Disasters](https://mlsanity.com/paper/1910.04416/) (2019)
- [Emotion-Based End-to-End Matching Between Image and Music in Valence-Arousal Space](https://mlsanity.com/paper/2009.05103/) (2020)
- [Visual Sentiment Prediction with Deep Convolutional Neural Networks](https://mlsanity.com/paper/1411.5731/) (2014)

## Built on by (most cited first, 30 of 47)

- [EmoSet: A Large-scale Visual Emotion Dataset with Rich Attributes](https://mlsanity.com/paper/2307.07961/) (2023)
- [MDAN: Multi-level Dependent Attention Network for Visual Emotion Analysis](https://mlsanity.com/paper/2203.13443/) (2022)
- [EmotiCrafter: Text-to-Emotional-Image Generation based on Valence-Arousal Model](https://mlsanity.com/paper/2501.05710/) (2025)
- [Emotion Recognition from Multiple Modalities: Fundamentals and Methodologies](https://mlsanity.com/paper/2108.10152/) (2021)
- [SOLVER: Scene-Object Interrelated Visual Emotion Reasoning Network](https://mlsanity.com/paper/2110.12334/) (2021)
- [SVFAP: Self-supervised Video Facial Affect Perceiver](https://mlsanity.com/paper/2401.00416/) (2023)
- [UniEmo: Unifying Emotional Understanding and Generation with Learnable Expert Queries](https://mlsanity.com/paper/2507.23372/) (2025)
- [Recursive Joint Cross-Modal Attention for Multimodal Fusion in Dimensional Emotion Recognition](https://mlsanity.com/paper/2403.13659/) (2024)
- [Dual-path Collaborative Generation Network for Emotional Video Captioning](https://mlsanity.com/paper/2408.03006/) (2024)
- [Unlocking the Emotional World of Visual Media: An Overview of the Science, Research, and Impact of Understanding Emotion](https://mlsanity.com/paper/2307.13463/) (2023)
- [LAKE-RED: Camouflaged Images Generation by Latent Background Knowledge Retrieval-Augmented Diffusion](https://mlsanity.com/paper/2404.00292/) (2024)
- [Affective Image Editing: Shaping Emotional Factors via Text Descriptions](https://mlsanity.com/paper/2505.18699/) (2025)
- [Learning to Compose Diversified Prompts for Image Emotion Classification](https://mlsanity.com/paper/2201.10963/) (2022)
- [CoEmoGen: Towards Semantically-Coherent and Scalable Emotional Image Content Generation](https://mlsanity.com/paper/2508.03535/) (2025)
- [Aesthetically Relevant Image Captioning](https://mlsanity.com/paper/2211.15378/) (2022)
- [Towards Deeper Emotional Reflection: Crafting Affective Image Filters with Generative Priors](https://mlsanity.com/paper/2512.17376/) (2025)
- [On the use of Vision-Language models for Visual Sentiment Analysis: a study on CLIP](https://mlsanity.com/paper/2310.12062/) (2023)
- [Affective Video Content Analysis: Decade Review and New Perspectives](https://mlsanity.com/paper/2310.17212/) (2023)
- [EmoFeedback2: Reinforcement of Continuous Emotional Image Generation via LVLM-based Reward and Textual Feedback](https://mlsanity.com/paper/2511.19982/) (2025)
- [Resolving Sentiment Discrepancy for Multimodal Sentiment Detection via Semantics Completion and Decomposition](https://mlsanity.com/paper/2407.07026/) (2024)
- [An Empirical Study on Configuring In-Context Learning Demonstrations for Unleashing MLLMs' Sentimental Perception Capability](https://mlsanity.com/paper/2505.16193/) (2025)
- [EmoCaliber: Advancing Reliable Visual Emotion Comprehension via Confidence Verbalization and Calibration](https://mlsanity.com/paper/2512.15528/) (2025)
- [Video-based Cross-modal Auxiliary Network for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2208.13954/) (2022)
- [Bodily expressed emotion understanding through integrating Laban movement analysis](https://mlsanity.com/paper/2304.02187/) (2023)
- [Towards Interpretability in Audio and Visual Affective Machine Learning: A Review](https://mlsanity.com/paper/2306.08933/) (2023)
- [Emotional Theory of Mind: Bridging Fast Visual Processing with Slow Linguistic Reasoning](https://mlsanity.com/paper/2310.19995/) (2023)
- [UniEmoX: Cross-modal Semantic-Guided Large-Scale Pretraining for Universal Scene Emotion Perception](https://mlsanity.com/paper/2409.18877/) (2024)
- [Fuzzy-aware Loss for Source-free Domain Adaptation in Visual Emotion Recognition](https://mlsanity.com/paper/2501.15519/) (2025)
- [Customizing Visual Emotion Evaluation for MLLMs: An Open-vocabulary, Multifaceted, and Scalable Approach](https://mlsanity.com/paper/2509.21950/) (2025)
- [Emotion-Director: Bridging Affective Shortcut in Emotion-Oriented Image Generation](https://mlsanity.com/paper/2512.19479/) (2025)

## Abstract (arXiv)

Images can convey rich semantics and induce various emotions in viewers. Recently, with the rapid advancement of emotional intelligence and the explosive growth of visual data, extensive research efforts have been dedicated to affective image content analysis (AICA). In this survey, we will comprehensively review the development of AICA in the recent two decades, especially focusing on the state-of-the-art methods with respect to three main challenges -- the affective gap, perception subjectivity, and label noise and absence. We begin with an introduction to the key emotion representation models that have been widely employed in AICA and description of available datasets for performing evaluation with quantitative comparison of label noise and dataset bias. We then summarize and compare the representative approaches on (1) emotion feature extraction, including both handcrafted and deep features, (2) learning methods on dominant emotion recognition, personalized emotion prediction, emotion distribution learning, and learning from noisy data or few labels, and (3) AICA based applications. Finally, we discuss some challenges and promising research directions in the future, such as image content and context understanding, group emotion clustering, and viewer-image interaction.