mlsanityData to 7 Oct 2026

From Local Concepts to Universals: Evaluating the Multicultural Understanding of Vision-Language Models

arXiv 2407.00263 ↗First posted 28 Jun 2024cs.CL cs.AI cs.CVcited by 14

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Despite recent advancements in vision-language models, their performance remains suboptimal on images from non-western cultures due to underrepresentation in training datasets. Various benchmarks have been proposed to test models' cultural inclusivity, but they have limited coverage of cultures and do not adequately assess cultural diversity across universal as well as culture-specific local concepts. To address these limitations, we introduce the GlobalRG benchmark, comprising two challenging tasks: retrieval across universals and cultural visual grounding. The former task entails retrieving culturally diverse images for universal concepts from 50 countries, while the latter aims at grounding culture-specific concepts within images from 15 countries. Our evaluation across a wide range of models reveals that the performance varies significantly across cultures -- underscoring the necessity for enhancing multicultural understanding in vision-language models.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 31 of its 58 references that are in the map. First posted on arXiv 28 Jun 2024.

Builds on

31 of 58 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond24 Aug
  2. 2023Kosmos-2: Grounding Multimodal Large Language Models to the World26 Jun
  3. 2022Vision-Language Pre-Training with Triple Contrastive Learning21 Feb

Other references · 28

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  4. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  5. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  6. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  7. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
  8. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,306
  9. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  10. 2016Modeling Context in Referring Expressionscited by 1,054
  11. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  12. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 646
16 more
  1. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 424
  2. 2022Challenges and Strategies in Cross-Cultural NLPcited by 216
  3. 2021Visually Grounded Reasoning across Languages and Culturescited by 188
  4. 2017No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing Worldcited by 152
  5. 2019Does Object Recognition Work for Everyone?cited by 149
  6. 2022Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervisioncited by 80
  7. 2022Extracting Cultural Commonsense Knowledge at Scalecited by 76
  8. 2021Broaden the Vision: Geo-Diverse Visual Commonsense Reasoningcited by 59
  9. 2023GeoDE: a Geographically Diverse Evaluation Dataset for Object Recognitioncited by 50
  10. 2023Multi-lingual and Multi-cultural Figurative Language Understandingcited by 47
  11. 2023Inspecting the Geographical Representativeness of Images from Text-to-Image Modelscited by 44
  12. 2022GRIT: General Robust Image Task Benchmarkcited by 26
  13. 2023GeoNet: Benchmarking Unsupervised Adaptation across Geographiescited by 17
  14. 2023GD-COMET: A Geo-Diverse Commonsense Inference Modelcited by 13
  15. 2021Towards A Fairer Landmark Recognition Datasetcited by 9
  16. 2024Annotations on a Budget: Leveraging Geo-Data Similarity to Balance Model Performance and Annotation Costcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 11; 2026: 2.

14 papers.

Sort:

By area:

  1. 2024Vision-Language Models under Cultural and Inclusive Considerationscited by 8
  2. 2025RAVENEA: A Benchmark for Multimodal Retrieval-Augmented Visual Culture Understandingcited by 8
  3. 2025Pearl: A Multimodal Culturally-Aware Arabic Instruction Datasetcited by 6
  4. 2025What are Foundation Models Cooking in the Post-Soviet World?cited by 3
  5. 2025CultureCLIP: Empowering CLIP with Cultural Awareness through Synthetic Images and Contextualized Captionscited by 3
  6. 2025Cultural Awareness in Vision-Language Models: A Cross-Country Explorationcited by 2
  7. 2025Culture In a Frame: C3B as a Comic-Based Benchmark for Multimodal Culturally Awarenesscited by 2
  8. 2025Toward Socially Aware Vision-Language Models: Evaluating Cultural Competence Through Multimodal Story Generationcited by 1
  9. 2025Rice-VL: Evaluating Vision-Language Models for Cultural Understanding Across ASEAN Countriescited by 1
  10. 2025Are Vision Language Models Cross-Cultural Theory of Mind Reasoners?cited by 1
  11. 2025From Perception to Punchline: Empowering VLM with the Art of In-the-wild Memecited by 1
  12. 2025CRAFT: Cultural Russian-Oriented Dataset Adaptation for Focused Text-to-Image Generationcited by 0
2 more
  1. 2026EgoGroups: A Benchmark For Detecting Social Groups of People in the Wildcited by 0
  2. 2026Beyond Semantics: Modeling Factual and Affective Perceptual Experiences from Vision-Language Datacited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024No Filter: Cultural and Socioeconomic Diversity in Contrastive Vision-Language Models
  2. 2024See It from My Perspective: How Language Affects Cultural Bias in Image Understanding
  3. 2025BLEnD-Vis: Benchmarking Multimodal Cultural Understanding in Vision Language Models
  4. 2025CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries
  5. 2024Benchmarking Vision Language Models for Cultural Understanding
  6. 2025Uncovering Cultural Representation Disparities in Vision-Language Models
  7. 2025Evaluation of Cultural Competence of Vision-Language Models
  8. 2025Vision Language Models are Confused Tourists