# ArtELingo: A Million Emotion Annotations of WikiArt with Emphasis on Diversity over Language and Culture

arXiv: https://arxiv.org/abs/2211.10780 · HTML: https://mlsanity.com/paper/2211.10780/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 57 references that are in the map.
- First posted on arXiv 19 Nov 2022.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [GoEmotions: A Dataset of Fine-Grained Emotions](https://mlsanity.com/paper/2005.00547/) (2020)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [Meshed-Memory Transformer for Image Captioning](https://mlsanity.com/paper/1912.08226/) (2019)
- [WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning](https://mlsanity.com/paper/2103.01913/) (2021)
- [VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioning](https://mlsanity.com/paper/2102.10407/) (2021)
- [Visually Grounded Reasoning across Languages and Cultures](https://mlsanity.com/paper/2109.13238/) (2021)
- [ArtEmis: Affective Language for Visual Art](https://mlsanity.com/paper/2101.07396/) (2021)
- [IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages](https://mlsanity.com/paper/2201.11732/) (2022)
- [It is Okay to Not Be Okay: Overcoming Emotional Bias in Affective Image Captioning by Contrastive Data Collection](https://mlsanity.com/paper/2204.07660/) (2022)
- [DENS: A Dataset for Multi-class Emotion Analysis](https://mlsanity.com/paper/1910.11769/) (2019)
- [Affect2MM: Affective Analysis of Multimedia Content Using Emotion Causality](https://mlsanity.com/paper/2103.06541/) (2021)
- [MLM: A Benchmark Dataset for Multitask Learning with Multiple Languages and Modalities](https://mlsanity.com/paper/2008.06376/) (2020)
- [What's in your Head? Emergent Behaviour in Multi-Task Transformer Models](https://mlsanity.com/paper/2104.06129/) (2021)
- [Gender Bias Hidden Behind Chinese Word Embeddings: The Case of Chinese Adjectives](https://mlsanity.com/paper/2106.00181/) (2021)

## Built on by (most cited first, 17 of 17)

- [HRS-Bench: Holistic, Reliable and Scalable Benchmark for Text-to-Image Models](https://mlsanity.com/paper/2304.05390/) (2023)
- [NLPositionality: Characterizing Design Biases of Datasets and Models](https://mlsanity.com/paper/2306.01943/) (2023)
- [Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Art](https://mlsanity.com/paper/2406.03930/) (2024)
- [See It from My Perspective: How Language Affects Cultural Bias in Image Understanding](https://mlsanity.com/paper/2406.11665/) (2024)
- [Fairness in Language Models Beyond English: Gaps and Challenges](https://mlsanity.com/paper/2302.12578/) (2023)
- [CULEMO: Cultural Lenses on Emotion -- Benchmarking LLMs for Cross-Cultural Emotion Understanding](https://mlsanity.com/paper/2503.10688/) (2025)
- [OxfordTVG-HIC: Can Machine Make Humorous Captions from Images?](https://mlsanity.com/paper/2307.11636/) (2023)
- [Semantic and Expressive Variation in Image Captions Across Languages](https://mlsanity.com/paper/2310.14356/) (2023)
- [Socratis: Are large multimodal models emotionally aware?](https://mlsanity.com/paper/2308.16741/) (2023)
- [Affective Visual Dialog: A Large-Scale Benchmark for Emotional Reasoning Based on Visually Grounded Conversations](https://mlsanity.com/paper/2308.16349/) (2023)
- [A survey of diversity quantification in natural language processing: The why, what, where and how](https://mlsanity.com/paper/2507.20858/) (2025)
- [An Efficient Watermarking Method for Latent Diffusion Models via Low-Rank Adaptation and Dynamic Loss Weighting](https://mlsanity.com/paper/2410.20202/) (2024)
- [The Algorithmic Gaze of Image Quality Assessment: An Audit and Trace Ethnography of the LAION-Aesthetics Predictor](https://mlsanity.com/paper/2601.09896/) (2026)
- [Solution for Emotion Prediction Competition of Workshop on Emotionally and Culturally Intelligent AI](https://mlsanity.com/paper/2403.17683/) (2024)
- [Pragmatics Meets Culture: Culturally-adapted Artwork Description Generation and Evaluation](https://mlsanity.com/paper/2604.02557/) (2026)
- [Dimensional Distribution Emotion State: Leveraging Valence and Arousal as a Common Embedding Space for Visual Emotion Analysis](https://mlsanity.com/paper/2605.26262/) (2026)
- [MUSE: Benchmarking Large Vision-Language Models on Multi-Modal Understanding in Situated Education](https://mlsanity.com/paper/2609.19088/) (2026)

## Abstract (arXiv)

This paper introduces ArtELingo, a new benchmark and dataset, designed to encourage work on diversity across languages and cultures. Following ArtEmis, a collection of 80k artworks from WikiArt with 0.45M emotion labels and English-only captions, ArtELingo adds another 0.79M annotations in Arabic and Chinese, plus 4.8K in Spanish to evaluate "cultural-transfer" performance. More than 51K artworks have 5 annotations or more in 3 languages. This diversity makes it possible to study similarities and differences across languages and cultures. Further, we investigate captioning tasks, and find diversity improves the performance of baseline models. ArtELingo is publicly available at https://www.artelingo.org/ with standard splits and baseline models. We hope our work will help ease future research on multilinguality and culturally-aware AI.