# Visual Affect Around the World: A Large-scale Multilingual Visual Sentiment Ontology

arXiv: https://arxiv.org/abs/1508.03868 · HTML: https://mlsanity.com/paper/1508.03868/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Aug 2015.

## Built on by (most cited first, 6 of 6)

- [Affective Image Content Analysis: Two Decades Review and New Perspectives](https://mlsanity.com/paper/2106.16125/) (2021)
- [Unlocking the Emotional World of Visual Media: An Overview of the Science, Research, and Impact of Understanding Emotion](https://mlsanity.com/paper/2307.13463/) (2023)
- [Automatic Expansion of Domain-Specific Affective Models for Web Intelligence Applications](https://mlsanity.com/paper/2102.00827/) (2021)
- [Learning Input-agnostic Manipulation Directions in StyleGAN with Text Guidance](https://mlsanity.com/paper/2302.13331/) (2023)
- [The Emotions of the Crowd: Learning Image Sentiment from Tweets via Cross-modal Distillation](https://mlsanity.com/paper/2304.14942/) (2023)
- [Survey of Generative Methods for Social Media Analysis](https://mlsanity.com/paper/2112.07041/) (2021)

## Abstract (arXiv)

Every culture and language is unique. Our work expressly focuses on the uniqueness of culture and language in relation to human affect, specifically sentiment and emotion semantics, and how they manifest in social multimedia. We develop sets of sentiment- and emotion-polarized visual concepts by adapting semantic structures called adjective-noun pairs, originally introduced by Borth et al. (2013), but in a multilingual context. We propose a new language-dependent method for automatic discovery of these adjective-noun constructs. We show how this pipeline can be applied on a social multimedia platform for the creation of a large-scale multilingual visual sentiment concept ontology (MVSO). Unlike the flat structure in Borth et al. (2013), our unified ontology is organized hierarchically by multilingual clusters of visually detectable nouns and subclusters of emotionally biased versions of these nouns. In addition, we present an image-based prediction task to show how generalizable language-specific models are in a multilingual context. A new, publicly available dataset of >15.6K sentiment-biased visual concepts across 12 languages with language-specific detector banks, >7.36M images and their metadata is also released.