mlsanityData to 7 Oct 2026

Navigating Cultural Chasms: Exploring and Unlocking the Cultural POV of Text-To-Image Models

arXiv 2310.01929 ↗First posted 3 Oct 2023cs.CL cs.AI cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

Text-To-Image (TTI) models, such as DALL-E and StableDiffusion, have demonstrated remarkable prompt-based image generation capabilities. Multilingual encoders may have a substantial impact on the cultural agency of these models, as language is a conduit of culture. In this study, we explore the cultural perception embedded in TTI models by characterizing culture across three hierarchical tiers: cultural dimensions, cultural domains, and cultural concepts. Based on this ontology, we derive prompt templates to unlock the cultural knowledge in TTI models, and propose a comprehensive suite of evaluation techniques, including intrinsic evaluations using the CLIP space, extrinsic evaluations with a Visual-Question-Answer (VQA) model and human assessments, to evaluate the cultural content of TTI-generated images. To bolster our research, we introduce the CulText2I dataset, derived from six diverse TTI models and spanning ten languages. Our experiments provide insights regarding Do, What, Which and How research questions about the nature of cultural encoding in TTI models, paving the way for cross-cultural applications of these models.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 40 of its 65 references that are in the map. First posted on arXiv 3 Oct 2023.

Builds on

40 of 65 references.

References · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  10. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  11. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  12. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
28 more
  1. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  2. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  3. 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 974
  4. 2021Transformer in Transformercited by 632
  5. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 329
  6. 2022DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Modelscited by 230
  7. 2023Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Studycited by 203
  8. 2021Visually Grounded Reasoning across Languages and Culturescited by 188
  9. 2022"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Datasetcited by 160
  10. 2023Having Beer after Prayer? Measuring Cultural Bias in Large Language Modelscited by 153
  11. 2022Probing Pre-Trained Language Models for Cross-Cultural Differences in Valuescited by 147
  12. 2023Social Biases through the Text-to-Image Generation Lenscited by 126
  13. 2023Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensionscited by 111
  14. 2021Mitigating Language-Dependent Ethnic Bias in BERTcited by 87
  15. 2021Larger-Scale Transformers for Multilingual Masked Language Modelingcited by 86
  16. 2022AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilitiescited by 84
  17. 2022StoryDALL-E: Adapting Pretrained Text-to-Image Transformers for Story Continuationcited by 74
  18. 2022Large-scale Text-to-Image Generation Models for Visual Artists' Creative Workscited by 60
  19. 2023Typology of Risks of Generative Text-to-Image Modelscited by 55
  20. 2023Knowledge of cultural moral norms in large language modelscited by 51
  21. 2023Inspecting the Geographical Representativeness of Images from Text-to-Image Modelscited by 44
  22. 2023The Hidden Language of Diffusion Modelscited by 34
  23. 2023AltDiffusion: A Multilingual Text-to-Image Diffusion Modelcited by 30
  24. 2022Exploiting Cultural Biases via Homoglyphs in Text-to-Image Synthesiscited by 29
  25. 2022Cultural Incongruencies in Artificial Intelligencecited by 21
  26. 2023Imitation versus Innovation: What children can do that large language and language-and-vision models cannot (yet)?cited by 13
  27. 2023On the Cultural Gap in Text-to-Image Generationcited by 10
  28. 2023Multilingual Conceptual Coverage in Text-to-Image Modelscited by 9

Built on by

Cited by year

Citing papers per year: 2024: 10; 2025: 6.

16 papers.

Sort:

By area:

  1. 2024Survey of Cultural Awareness in Language Models: Text and Beyondcited by 101
  2. 2024VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Modelscited by 42
  3. 2024An image speaks a thousand words, but can everyone listen? On image transcreation for cultural relevancecited by 36
  4. 2024Beyond Aesthetics: Cultural Competence in Text-to-Image Modelscited by 30
  5. 2024Are LLMs Better than Reported? Detecting Label Errors and Mitigating Their Effect on Model Performancecited by 20
  6. 2025Beyond Words: Exploring Cultural Value Sensitivity in Multimodal Modelscited by 14
  7. 2024Vision-Language Models under Cultural and Inclusive Considerationscited by 8
  8. 2025CuRe: Cultural Gaps in the Long Tail of Text-to-Image Systemscited by 8
  9. 2024No Longer Trending on Artstation: Prompt Analysis of Generative AI Artcited by 7
  10. 2024NL-Eye: Abductive NLI for Imagescited by 5
  11. 2024Uplifting Lower-Income Data: Strategies for Socioeconomic Perspective Shifts in Large Multi-modal Modelscited by 2
  12. 2024Locally Measuring Cross-lingual Lexical Alignment: A Domain and Word Level Perspectivecited by 1
4 more
  1. 2025Synthetic History: Evaluating Visual Representations of the Past in Diffusion Modelscited by 1
  2. 2025CAIRe: Cultural Attribution of Images by Retrieval-Augmented Evaluationcited by 1
  3. 2025Text-to-Image Models and Their Representation of People from Different Nationalities Engaging in Activitiescited by 0
  4. 2025The Persistence of Cultural Memory: Investigating Multimodal Iconicity in Diffusion Modelscited by 0