mlsanityData to 7 Oct 2026

Crossroads of Continents: Automated Artifact Extraction for Cultural Adaptation with Large Multimodal Models

arXiv 2407.02067 ↗First posted 2 Jul 2024cs.CLcited by 2

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

We present a comprehensive three-phase study to examine (1) the cultural understanding of Large Multimodal Models (LMMs) by introducing DalleStreet, a large-scale dataset generated by DALL-E 3 and validated by humans, containing 9,935 images of 67 countries and 10 concept classes; (2) the underlying implicit and potentially stereotypical cultural associations with a cultural artifact extraction task; and (3) an approach to adapt cultural representation in an image based on extracted associations using a modular pipeline, CultureAdapt. We find disparities in cultural understanding at geographic sub-region levels with both open-source (LLaVA) and closed-source (GPT-4V) models on DalleStreet and other existing benchmarks, which we try to understand using over 18,000 artifacts that we identify in association to different countries. Our findings reveal a nuanced picture of the cultural competence of LMMs, highlighting the need to develop culture-aware systems. Dataset and code are available at https://github.com/iamshnoo/crossroads

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 32 of its 46 references that are in the map. First posted on arXiv 2 Jul 2024.

Builds on

32 of 46 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024No Filter: Cultural and Socioeconomic Diversity in Contrastive Vision-Language Models22 May
  2. 2024Exploring Visual Culture Awareness in GPT-4V: A Comprehensive Probing8 Feb
  3. 2023Tag2Text: Guiding Vision-Language Model via Image Tagging10 Mar
  4. 2023GIVL: Improving Geographical Inclusivity of Vision-Language Models with Pre-Training Methods5 Jan

Other references · 28

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  6. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  7. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  8. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  9. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  10. 2019Quantifying the Carbon Emissions of Machine Learningcited by 516
  11. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  12. 2023Recognize Anything: A Strong Image Tagging Modelcited by 305
16 more
  1. 2023TRAK: Attributing Model Behavior at Scalecited by 273
  2. 2022Challenges and Strategies in Cross-Cultural NLPcited by 216
  3. 2021Visually Grounded Reasoning across Languages and Culturescited by 188
  4. 2024Towards Measuring and Modeling "Culture" in LLMs: A Surveycited by 166
  5. 2013Stabilitycited by 119
  6. 2024Zero-shot Image Editing with Reference Imitationcited by 64
  7. 2024Massively Multi-Cultural Knowledge Acquisition & LM Benchmarkingcited by 63
  8. 2024An image speaks a thousand words, but can everyone listen? On image transcreation for cultural relevancecited by 36
  9. 2024ViSAGe: A Global-Scale Analysis of Visual Stereotypes in Text-to-Image Generationcited by 27
  10. 2024Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Artcited by 26
  11. 2024How Culture Shapes What People Want From AIcited by 23
  12. 2024BertaQA: How Much Do Language Models Know About Local Culture?cited by 23
  13. 2024VALOR-EVAL: Holistic Coverage and Faithfulness Evaluation of Large Vision-Language Modelscited by 16
  14. 2024Generalized People Diversity: Learning a Human Perception-Aligned Diversity Representation for People Imagescited by 6
  15. 2019Modeling Color Terminology Across Thousands of Languagescited by 5
  16. 2024Coimagining the Future of Voice Assistants with Cultural Sensitivitycited by 3

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2025Beyond Words: Exploring Cultural Value Sensitivity in Multimodal Modelscited by 14
  2. 2024Vision-Language Models under Cultural and Inclusive Considerationscited by 8

Similar, not cited

Close in wording, no citation either way

  1. 2025Diffusion Models Through a Global Lens: Are They Culturally Inclusive?
  2. 2025Culture in Action: Evaluating Text-to-Image Models through Social Activities
  3. 2025CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries
  4. 2025Seeing Culture: A Benchmark for Visual Reasoning and Grounding
  5. 2025Culture In a Frame: C3B as a Comic-Based Benchmark for Multimodal Culturally Awareness
  6. 2025Hanfu-Bench: A Multimodal Benchmark on Cross-Temporal Cultural Understanding and Transcreation
  7. 2026Appear2Meaning: A Cross-Cultural Benchmark for Structured Cultural Metadata Inference from Images
  8. 2024The Power of Many: Multi-Agent Multimodal Models for Cultural Image Captioning