# Senti-Attend: Image Captioning using Sentiment and Attention

arXiv: https://arxiv.org/abs/1811.09789 · HTML: https://mlsanity.com/paper/1811.09789/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Nov 2018.

## Built on by (most cited first, 5 of 5)

- [I Can't Believe There's No Images! Learning Visual Tasks Using only Language Supervision](https://mlsanity.com/paper/2211.09778/) (2022)
- [Large Language Models Meet Text-Centric Multimodal Sentiment Analysis: A Survey](https://mlsanity.com/paper/2406.08068/) (2024)
- [Diverse Image Captioning with Grounded Style](https://mlsanity.com/paper/2205.01813/) (2022)
- [How to Describe Images in a More Funny Way? Towards a Modular Approach to Cross-Modal Sarcasm Generation](https://mlsanity.com/paper/2211.10992/) (2022)
- [FaceGemma: Enhancing Image Captioning with Facial Attributes for Portrait Images](https://mlsanity.com/paper/2309.13601/) (2023)

## Abstract (arXiv)

There has been much recent work on image captioning models that describe the factual aspects of an image. Recently, some models have incorporated non-factual aspects into the captions, such as sentiment or style. However, such models typically have difficulty in balancing the semantic aspects of the image and the non-factual dimensions of the caption; in addition, it can be observed that humans may focus on different aspects of an image depending on the chosen sentiment or style of the caption. To address this, we design an attention-based model to better add sentiment to image captions. The model embeds and learns sentiment with respect to image-caption data, and uses both high-level and word-level sentiment information during the learning process. The model outperforms the state-of-the-art work in image captioning with sentiment using standard evaluation metrics. An analysis of generated captions also shows that our model does this by a better selection of the sentiment-bearing adjectives and adjective-noun pairs.