# Navigating Cultural Chasms: Exploring and Unlocking the Cultural POV of Text-To-Image Models

arXiv: https://arxiv.org/abs/2310.01929 · HTML: https://mlsanity.com/paper/2310.01929/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 65 references that are in the map.
- First posted on arXiv 3 Oct 2023.

## Builds on (most cited first, 30 of 40)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Reproducible scaling laws for contrastive language-image learning](https://mlsanity.com/paper/2212.07143/) (2022)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [Transformer in Transformer](https://mlsanity.com/paper/2103.00112/) (2021)
- [Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery](https://mlsanity.com/paper/2302.03668/) (2023)
- [DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models](https://mlsanity.com/paper/2202.04053/) (2022)
- [Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study](https://mlsanity.com/paper/2303.17466/) (2023)
- [Visually Grounded Reasoning across Languages and Cultures](https://mlsanity.com/paper/2109.13238/) (2021)
- ["I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Dataset](https://mlsanity.com/paper/2205.09209/) (2022)
- [Having Beer after Prayer? Measuring Cultural Bias in Large Language Models](https://mlsanity.com/paper/2305.14456/) (2023)
- [Probing Pre-Trained Language Models for Cross-Cultural Differences in Values](https://mlsanity.com/paper/2203.13722/) (2022)
- [Social Biases through the Text-to-Image Generation Lens](https://mlsanity.com/paper/2304.06034/) (2023)
- [Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensions](https://mlsanity.com/paper/2309.12342/) (2023)
- [Larger-Scale Transformers for Multilingual Masked Language Modeling](https://mlsanity.com/paper/2105.00572/) (2021)
- [Mitigating Language-Dependent Ethnic Bias in BERT](https://mlsanity.com/paper/2109.05704/) (2021)
- [AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilities](https://mlsanity.com/paper/2211.06679/) (2022)
- [StoryDALL-E: Adapting Pretrained Text-to-Image Transformers for Story Continuation](https://mlsanity.com/paper/2209.06192/) (2022)
- [Large-scale Text-to-Image Generation Models for Visual Artists' Creative Works](https://mlsanity.com/paper/2210.08477/) (2022)

## Built on by (most cited first, 16 of 16)

- [Survey of Cultural Awareness in Language Models: Text and Beyond](https://mlsanity.com/paper/2411.00860/) (2024)
- [VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2411.13503/) (2024)
- [An image speaks a thousand words, but can everyone listen? On image transcreation for cultural relevance](https://mlsanity.com/paper/2404.01247/) (2024)
- [Beyond Aesthetics: Cultural Competence in Text-to-Image Models](https://mlsanity.com/paper/2407.06863/) (2024)
- [Are LLMs Better than Reported? Detecting Label Errors and Mitigating Their Effect on Model Performance](https://mlsanity.com/paper/2410.18889/) (2024)
- [Beyond Words: Exploring Cultural Value Sensitivity in Multimodal Models](https://mlsanity.com/paper/2502.14906/) (2025)
- [Vision-Language Models under Cultural and Inclusive Considerations](https://mlsanity.com/paper/2407.06177/) (2024)
- [CuRe: Cultural Gaps in the Long Tail of Text-to-Image Systems](https://mlsanity.com/paper/2506.08071/) (2025)
- [No Longer Trending on Artstation: Prompt Analysis of Generative AI Art](https://mlsanity.com/paper/2401.14425/) (2024)
- [NL-Eye: Abductive NLI for Images](https://mlsanity.com/paper/2410.02613/) (2024)
- [Uplifting Lower-Income Data: Strategies for Socioeconomic Perspective Shifts in Large Multi-modal Models](https://mlsanity.com/paper/2407.02623/) (2024)
- [Locally Measuring Cross-lingual Lexical Alignment: A Domain and Word Level Perspective](https://mlsanity.com/paper/2410.07239/) (2024)
- [Synthetic History: Evaluating Visual Representations of the Past in Diffusion Models](https://mlsanity.com/paper/2505.17064/) (2025)
- [CAIRe: Cultural Attribution of Images by Retrieval-Augmented Evaluation](https://mlsanity.com/paper/2506.09109/) (2025)
- [Text-to-Image Models and Their Representation of People from Different Nationalities Engaging in Activities](https://mlsanity.com/paper/2504.06313/) (2025)
- [The Persistence of Cultural Memory: Investigating Multimodal Iconicity in Diffusion Models](https://mlsanity.com/paper/2511.11435/) (2025)

## Abstract (arXiv)

Text-To-Image (TTI) models, such as DALL-E and StableDiffusion, have demonstrated remarkable prompt-based image generation capabilities. Multilingual encoders may have a substantial impact on the cultural agency of these models, as language is a conduit of culture. In this study, we explore the cultural perception embedded in TTI models by characterizing culture across three hierarchical tiers: cultural dimensions, cultural domains, and cultural concepts. Based on this ontology, we derive prompt templates to unlock the cultural knowledge in TTI models, and propose a comprehensive suite of evaluation techniques, including intrinsic evaluations using the CLIP space, extrinsic evaluations with a Visual-Question-Answer (VQA) model and human assessments, to evaluate the cultural content of TTI-generated images. To bolster our research, we introduce the CulText2I dataset, derived from six diverse TTI models and spanning ten languages. Our experiments provide insights regarding Do, What, Which and How research questions about the nature of cultural encoding in TTI models, paving the way for cross-cultural applications of these models.