# SOLVER: Scene-Object Interrelated Visual Emotion Reasoning Network

arXiv: https://arxiv.org/abs/2110.12334 · HTML: https://mlsanity.com/paper/2110.12334/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 93 references that are in the map.
- First posted on arXiv 24 Oct 2021.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Gated Graph Sequence Neural Networks](https://mlsanity.com/paper/1511.05493/) (2015)
- [Scene Graph Generation by Iterative Message Passing](https://mlsanity.com/paper/1701.02426/) (2017)
- [Multi-Label Image Recognition with Graph Convolutional Networks](https://mlsanity.com/paper/1904.03582/) (2019)
- [Relation Networks for Object Detection](https://mlsanity.com/paper/1711.11575/) (2017)
- [Graph R-CNN for Scene Graph Generation](https://mlsanity.com/paper/1808.00191/) (2018)
- [Visual Semantic Reasoning for Image-Text Matching](https://mlsanity.com/paper/1909.02701/) (2019)
- [Graph-Based Global Reasoning Networks](https://mlsanity.com/paper/1811.12814/) (2018)
- [Building a Large Scale Dataset for Image Emotion Recognition: The Fine Print and The Benchmark](https://mlsanity.com/paper/1605.02677/) (2016)
- [Affective Image Content Analysis: Two Decades Review and New Perspectives](https://mlsanity.com/paper/2106.16125/) (2021)
- [Robust Image Sentiment Analysis Using Progressively Trained and Domain Transferred Deep Networks](https://mlsanity.com/paper/1509.06041/) (2015)
- [Learning Multi-level Deep Representations for Image Emotion Classification](https://mlsanity.com/paper/1611.07145/) (2016)
- ["Factual" or "Emotional": Stylized Image Captioning with Adaptive Learning and Attention](https://mlsanity.com/paper/1807.03871/) (2018)
- [Stimuli-Aware Visual Emotion Analysis](https://mlsanity.com/paper/2109.01812/) (2021)
- [Learning Social Relation Traits from Face Images](https://mlsanity.com/paper/1509.03936/) (2015)
- [DeepSentiBank: Visual Sentiment Concept Classification with Deep Convolutional Neural Networks](https://mlsanity.com/paper/1410.8586/) (2014)
- [Temporal Reasoning Graph for Activity Recognition](https://mlsanity.com/paper/1908.09995/) (2019)
- [Affective Computing for Large-Scale Heterogeneous Multimedia Data: A Survey](https://mlsanity.com/paper/1911.05609/) (2019)
- [A Circular-Structured Representation for Visual Emotion Distribution Learning](https://mlsanity.com/paper/2106.12450/) (2021)

## Built on by (most cited first, 18 of 18)

- [EmoSet: A Large-scale Visual Emotion Dataset with Rich Attributes](https://mlsanity.com/paper/2307.07961/) (2023)
- [EmoGen: Emotional Image Content Generation with Text-to-Image Diffusion Models](https://mlsanity.com/paper/2401.04608/) (2024)
- [EmoEdit: Evoking Emotions through Image Manipulation](https://mlsanity.com/paper/2405.12661/) (2024)
- [UniEmo: Unifying Emotional Understanding and Generation with Learnable Expert Queries](https://mlsanity.com/paper/2507.23372/) (2025)
- [Dual-path Collaborative Generation Network for Emotional Video Captioning](https://mlsanity.com/paper/2408.03006/) (2024)
- [Unlocking the Emotional World of Visual Media: An Overview of the Science, Research, and Impact of Understanding Emotion](https://mlsanity.com/paper/2307.13463/) (2023)
- [Learning to Compose Diversified Prompts for Image Emotion Classification](https://mlsanity.com/paper/2201.10963/) (2022)
- [Seeking Subjectivity in Visual Emotion Distribution Learning](https://mlsanity.com/paper/2207.11875/) (2022)
- [EmoCtrl: Controllable Emotional Image Content Generation](https://mlsanity.com/paper/2512.22437/) (2025)
- [EmoKGEdit: Training-free Affective Injection via Visual Cue Transformation](https://mlsanity.com/paper/2601.12326/) (2026)
- [UniEmoX: Cross-modal Semantic-Guided Large-Scale Pretraining for Universal Scene Emotion Perception](https://mlsanity.com/paper/2409.18877/) (2024)
- [Fuzzy-aware Loss for Source-free Domain Adaptation in Visual Emotion Recognition](https://mlsanity.com/paper/2501.15519/) (2025)
- [Don't Judge Before You CLIP: A Unified Approach for Perceptual Tasks](https://mlsanity.com/paper/2503.13260/) (2025)
- [EmoSEM: Segment and Explain Emotion Stimuli in Visual Art](https://mlsanity.com/paper/2504.14658/) (2025)
- [Read-Best Is Not Steer-Best: A Probing--Steering Layer Dissociation in Omni-Modal Large Language Models](https://mlsanity.com/paper/2609.22135/) (2026)
- [Analyzing Image Beyond Visual Aspect: Image Emotion Classification via Multiple-Affective Captioning](https://mlsanity.com/paper/2511.23115/) (2025)
- [Emotional regulation improves deep learning-based image classification](https://mlsanity.com/paper/2606.13081/) (2026)
- [Balancing Emotional Alignment and Semantic Consistency in Image Generation via Reinforcement Learning with Valence-Arousal Anchoring](https://mlsanity.com/paper/2609.12830/) (2026)

## Abstract (arXiv)

Visual Emotion Analysis (VEA) aims at finding out how people feel emotionally towards different visual stimuli, which has attracted great attention recently with the prevalence of sharing images on social networks. Since human emotion involves a highly complex and abstract cognitive process, it is difficult to infer visual emotions directly from holistic or regional features in affective images. It has been demonstrated in psychology that visual emotions are evoked by the interactions between objects as well as the interactions between objects and scenes within an image. Inspired by this, we propose a novel Scene-Object interreLated Visual Emotion Reasoning network (SOLVER) to predict emotions from images. To mine the emotional relationships between distinct objects, we first build up an Emotion Graph based on semantic concepts and visual features. Then, we conduct reasoning on the Emotion Graph using Graph Convolutional Network (GCN), yielding emotion-enhanced object features. We also design a Scene-Object Fusion Module to integrate scenes and objects, which exploits scene features to guide the fusion process of object features with the proposed scene-based attention mechanism. Extensive experiments and comparisons are conducted on eight public visual emotion datasets, and the results demonstrate that the proposed SOLVER consistently outperforms the state-of-the-art methods by a large margin. Ablation studies verify the effectiveness of our method and visualizations prove its interpretability, which also bring new insight to explore the mysteries in VEA. Notably, we further discuss SOLVER on three other potential datasets with extended experiments, where we validate the robustness of our method and notice some limitations of it.