# When Saliency Meets Sentiment: Understanding How Image Content Invokes Emotion and Sentiment

arXiv: https://arxiv.org/abs/1611.04636 · HTML: https://mlsanity.com/paper/1611.04636/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Nov 2016.

## Built on by (most cited first, 4 of 4)

- [Holistic Visual-Textual Sentiment Analysis with Prior Models](https://mlsanity.com/paper/2211.12981/) (2022)
- [On the Complementarity of Images and Text for the Expression of Emotions in Social Media](https://mlsanity.com/paper/2202.07427/) (2022)
- [EmoSEM: Segment and Explain Emotion Stimuli in Visual Art](https://mlsanity.com/paper/2504.14658/) (2025)
- [Evaluating authenticity and quality of image captions via sentiment and semantic analyses](https://mlsanity.com/paper/2409.09560/) (2024)

## Abstract (arXiv)

Sentiment analysis is crucial for extracting social signals from social media content. Due to the prevalence of images in social media, image sentiment analysis is receiving increasing attention in recent years. However, most existing systems are black-boxes that do not provide insight on how image content invokes sentiment and emotion in the viewers. Psychological studies have confirmed that salient objects in an image often invoke emotions. In this work, we investigate more fine-grained and more comprehensive interaction between visual saliency and visual sentiment. In particular, we partition images in several primary scene-type dimensions, including: open-closed, natural-manmade, indoor-outdoor, and face-noface. Using state of the art saliency detection algorithm and sentiment classification algorithm, we examine how the sentiment of the salient region(s) in an image relates to the overall sentiment of the image. The experiments on a representative image emotion dataset have shown interesting correlation between saliency and sentiment in different scene types and in turn shed light on the mechanism of visual sentiment evocation.