mlsanityData to 14 Oct 2026

No Culture Left Behind: ArtELingo-28, a Benchmark of WikiArt with Captions in 28 Languages

arXiv 2411.03769 ↗First posted 6 Nov 2024cs.CL cs.AI cs.CYcited by 19

Not on the map. Why?

Abstract (arXiv)

Research in vision and language has made considerable progress thanks to benchmarks such as COCO. COCO captions focused on unambiguous facts in English; ArtEmis introduced subjective emotions and ArtELingo introduced some multilinguality (Chinese and Arabic). However we believe there should be more multilinguality. Hence, we present ArtELingo-28, a vision-language benchmark that spans 28 languages and encompasses approximately 200,000 annotations (140 annotations per image). Traditionally, vision research focused on unambiguous class labels, whereas ArtELingo-28 emphasizes diversity of opinions over languages and cultures. The challenge is to build machine learning systems that assign emotional captions to images. Baseline results will be presented for three novel conditions: Zero-Shot, Few-Shot and One-vs-All Zero-Shot. We find that cross-lingual transfer is more successful for culturally-related languages. Data and code are provided at www.artelingo.org.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 37 of its 70 references that are in the map. First posted on arXiv 6 Nov 2024.

Builds on

37 of 70 references.

References · 37

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  5. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  6. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  7. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  8. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  9. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  10. 2015VQA: Visual Question Answeringcited by 2,506
  11. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  12. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
25 more
  1. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
  2. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  3. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  4. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  5. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,362
  6. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  7. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 670
  8. 2021ClipCap: CLIP Prefix for Image Captioningcited by 559
  9. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 559
  10. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 556
  11. 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 486
  12. 2016Multi30K: Multilingual English-German Image Descriptionscited by 250
  13. 2023A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPTcited by 224
  14. 2021VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioningcited by 191
  15. 2021Visually Grounded Reasoning across Languages and Culturescited by 189
  16. 2021ArtEmis: Affective Language for Visual Artcited by 104
  17. 2018COCO-CN for Cross-Lingual Image Tagging, Captioning and Retrievalcited by 80
  18. 2023Towards Language Models That Can See: Computer Vision Through the LENS of Natural Languagecited by 53
  19. 2017STAIR Captions: Constructing a Large-Scale Japanese Image Caption Datasetcited by 47
  20. 2023mBLIP: Efficient Bootstrapping of Multilingual Vision-LLMscited by 41
  21. 2022It is Okay to Not Be Okay: Overcoming Emotional Bias in Affective Image Captioning by Contrastive Data Collectioncited by 36
  22. 2022Cross-View Language Modeling: Towards Unified Cross-Lingual Cross-Modal Pre-trainingcited by 24
  23. 2015Bridge Correlational Neural Networks for Multilingual Multimodal Representation Learningcited by 18
  24. 2019DENS: A Dataset for Multi-class Emotion Analysiscited by 17
  25. 2023Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Taskscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 10; 2026: 9.

19 papers.

Sort:

By area:

  1. 2025The Bitter Lesson Learned from 2,000+ Multilingual Benchmarkscited by 31
  2. 2025BRIGHTER: BRIdging the Gap in Human-Annotated Textual Emotion Recognition Datasets for 28 Languagescited by 27
  3. 2025SemEval-2025 Task 11: Bridging the Gap in Text-Based Emotion Detectioncited by 9
  4. 2026Translation as a Scalable Proxy for Multilingual Evaluationcited by 6
  5. 2026Tears or Cheers? Benchmarking LLMs via Culturally Elicited Distinct Affective Responsescited by 6
  6. 2025Uncovering Cultural Representation Disparities in Vision-Language Modelscited by 4
  7. 2025Evaluation of Cultural Competence of Vision-Language Modelscited by 3
  8. 2025World in a Frame: Understanding Culture Mixing as a New Challenge for Vision-Language Modelscited by 3
  9. 2025Exploring Cultural Nuances in Emotion Perception Across 15 African Languagescited by 1
  10. 2025Multilingual Vision-Language Models, A Surveycited by 1
  11. 2026CArtBench: Evaluating Vision-Language Models on Chinese Art Understanding, Interpretation, and Authenticitycited by 1
  12. 2025Comparable Corpora: Opportunities for New Research Directionscited by 0
7 more
  1. 2025AFRICAPTION: Establishing a New Paradigm for Image Captioning in African Languagescited by 0
  2. 2026CiQi-Agent: Aligning Vision, Tools and Aesthetics in Multimodal Agent for Cultural Reasoning on Chinese Porcelainscited by 0
  3. 2026Pragmatics Meets Culture: Culturally-adapted Artwork Description Generation and Evaluationcited by 0
  4. 2026Beyond Semantics: Modeling Factual and Affective Perceptual Experiences from Vision-Language Datacited by 0
  5. 2026ArtECulture: Benchmarking Culture-Conditioned Visual Emotion Understanding in Multimodal Large Language Modelscited by 0
  6. 2026CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systemscited by 0
  7. 2026FineWeb-CLaR: Culture, Language, and Region Annotations for Benchmark-Aligned Corpus Auditingcited by 0