Do Generative AI Models Output Harm while Representing Non-Western Cultures: Evidence from A Community-Centered Approach
Not on the map. Why?
Abstract (arXiv)
Our research investigates the impact of Generative Artificial Intelligence (GAI) models, specifically text-to-image generators (T2Is), on the representation of non-Western cultures, with a focus on Indian contexts. Despite the transformative potential of T2Is in content creation, concerns have arisen regarding biases that may lead to misrepresentations and marginalizations. Through a community-centered approach and grounded theory analysis of 5 focus groups from diverse Indian subcultures, we explore how T2I outputs to English prompts depict Indian culture and its subcultures, uncovering novel representational harms such as exoticism and cultural misappropriation. These findings highlight the urgent need for inclusive and culturally sensitive T2I systems. We propose design guidelines informed by a sociotechnical perspective, aiming to address these issues and contribute to the development of more equitable and representative GAI technologies globally. Our work also underscores the necessity of adopting a community-centered approach to comprehend the sociotechnical dynamics of these models, complementing existing work in this space while identifying and addressing the potential negative repercussions and harms that may arise when these models are deployed on a global scale.
Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 18 of its 85 references that are in the map. First posted on arXiv 20 Jul 2024.
Builds on
18 of 85 references.
References · 18
- 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
- 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
- 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
- 2019On Measuring and Mitigating Biased Inferences of Word Embeddingscited by 112
- 2021Re-imagining Algorithmic Fairness in India and Beyondcited by 102
- 2020Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligencecited by 100
- 2021On Measures of Biases and Harms in NLPcited by 81
- 2021A Survey of Race, Racism, and Anti-Racism in NLPcited by 74
- 2023AI's Regimes of Representation: A Community-centered Study of Text-to-Image Models in South Asiacited by 54
- 2023WEIRD FAccTs: How Western, Educated, Industrialized, Rich, and Democratic is FAccT?cited by 28
- 2023Sociodemographic Bias in Language Models: A Survey and Forward Pathcited by 27
- 2023Automated Ableism: An Exploration of Explicit Disability Biases in Sentiment and Toxicity Analysis Modelscited by 20
6 more
- 2024An Audit on the Perspectives and Challenges of Hallucinations in NLPcited by 17
- 2023Unmasking Nationality Bias: A Study of Human Perception of Nationalities in AI-Generated Articlescited by 16
- 2024The "Colonial Impulse" of Natural Language Processing: An Audit of Bengali Sentiment Analysis Tools and Their Identity-based Biasescited by 16
- 2023Out of Context: Investigating the Bias and Fairness Concerns of "Artificial Intelligence as a Service"cited by 15
- 2023CALM : A Multi-task Benchmark for Comprehensive Assessment of Language Model Biascited by 13
- 2023The Sentiment Problem: A Critical Survey towards Deconstructing Sentiment Analysiscited by 12
Built on by
Citing papers per year: 2024: 3; 2025: 12; 2026: 10.
25 papers.
Sort:
By area:
- 2025Position is Power: System Prompts as a Mechanism of Bias in Large Language Models (LLMs)cited by 19
- 2025The Case for "Thick Evaluations" of Cultural Representation in AIcited by 16
- 2024Search Engines in an AI Era: The False Promise of Factual and Verifiable Source-Cited Responsescited by 15
- 2025Can LLMs Express Personality Across Cultures? Introducing CulturalPersonas for Evaluating Trait Alignmentcited by 9
- 2025A Tale of Two Identities: An Ethical Audit of Human and AI-Crafted Personascited by 8
- 2024Hey GPT, Can You be More Racist? Analysis from Crowdsourced Attempts to Elicit Biased Content from Generative AIcited by 6
- 2025Invisible Filters: Cultural Bias in Hiring Evaluations Using Large Language Modelscited by 6
- 2024KAHANI: Culturally-Nuanced Visual Storytelling Tool for Non-Western Culturescited by 4
- 2025Through the Prism of Culture: Evaluating LLMs' Understanding of Indian Subcultures and Traditionscited by 3
- 2025RusCode: Russian Cultural Code Benchmark for Text-to-Image Generationcited by 1
- 2025What Is the Point of Equality in Machine Learning Fairness? Beyond Equality of Opportunitycited by 1
- 2026Going PLACES: Participatory Localized Red Teaming for Text-to-Image Safety in the Global Southcited by 1
13 more
- 2026Prompt Revision as a Source of Cultural Bias in Text-to-Image Systemscited by 1
- 2025Sometimes the Model doth Preach: Quantifying Religious Bias in Open LLMs through Demographic Analysis in Asian Nationscited by 0
- 2025Text-to-Image Models and Their Representation of People from Different Nationalities Engaging in Activitiescited by 0
- 2025NLP Meets the World: Toward Improving Conversations With the Public About Natural Language Processing Researchcited by 0
- 2025Designing and Evaluating Chain-of-Hints for Scientific Question Answeringcited by 0
- 2026Responsible AI for General-Purpose Systems: Overview, Challenges, and A Path Forwardcited by 0
- 2026MAPLE: Metadata Conditioned LLM Pretraining for Locale-Aware Question Answeringcited by 0
- 2026Operationalizing Fairness in Text-to-Image Models: A Survey of Bias, Fairness Audits and Mitigation Strategiescited by 0
- 2026Mod-Guide: An LLM-based Content Moderation Feedback System to Address Insensitive Speech toward Indigenous Ethnic and Religious Minority Communitiescited by 0
- 2026The Market in the Model: Latent Diffusion as Neural Economycited by 0
- 2026The Crowded Embedding Space: A Mean-Field Mechanism for Emergent Marginalization in Retrieval-Augmented Agentscited by 0
- 2026Harm is not Universal: Community-Specific Toxicity Detection is Urgently Neededcited by 0
- 2026Identifying Harm in Personalized, Generative AI Systems Requires User-Centered Auditing at the Interaction Levelcited by 0
- 2026Prompt Revision as a Source of Cultural Bias in Text-to-Image Systems10 Sep 2026
- 2026Identifying Harm in Personalized, Generative AI Systems Requires User-Centered Auditing at the Interaction Level7 Aug 2026
- 2026Harm is not Universal: Community-Specific Toxicity Detection is Urgently Needed27 Jul 2026
- 2026The Market in the Model: Latent Diffusion as Neural Economy17 Jun 2026
- 2026Mod-Guide: An LLM-based Content Moderation Feedback System to Address Insensitive Speech toward Indigenous Ethnic and Religious Minority Communities11 Jun 2026
- 2026The Crowded Embedding Space: A Mean-Field Mechanism for Emergent Marginalization in Retrieval-Augmented Agents1 Jun 2026
- 2026Going PLACES: Participatory Localized Red Teaming for Text-to-Image Safety in the Global South18 May 2026
- 2026Operationalizing Fairness in Text-to-Image Models: A Survey of Bias, Fairness Audits and Mitigation Strategies15 Apr 2026
- 2026MAPLE: Metadata Conditioned LLM Pretraining for Locale-Aware Question Answering21 Jan 2026
- 2026Responsible AI for General-Purpose Systems: Overview, Challenges, and A Path Forward19 Jan 2026
- 2025Designing and Evaluating Chain-of-Hints for Scientific Question Answering24 Oct 2025
- 2025Invisible Filters: Cultural Bias in Hiring Evaluations Using Large Language Models21 Aug 2025
13 more
- 2025NLP Meets the World: Toward Improving Conversations With the Public About Natural Language Processing Research2 Jul 2025
- 2025What Is the Point of Equality in Machine Learning Fairness? Beyond Equality of Opportunity20 Jun 2025
- 2025Can LLMs Express Personality Across Cultures? Introducing CulturalPersonas for Evaluating Trait Alignment6 Jun 2025
- 2025Position is Power: System Prompts as a Mechanism of Bias in Large Language Models (LLMs)27 May 2025
- 2025A Tale of Two Identities: An Ethical Audit of Human and AI-Crafted Personas7 May 2025
- 2025Text-to-Image Models and Their Representation of People from Different Nationalities Engaging in Activities8 Apr 2025
- 2025The Case for "Thick Evaluations" of Cultural Representation in AI24 Mar 2025
- 2025Sometimes the Model doth Preach: Quantifying Religious Bias in Open LLMs through Demographic Analysis in Asian Nations10 Mar 2025
- 2025RusCode: Russian Cultural Code Benchmark for Text-to-Image Generation11 Feb 2025
- 2025Through the Prism of Culture: Evaluating LLMs' Understanding of Indian Subcultures and Traditions28 Jan 2025
- 2024KAHANI: Culturally-Nuanced Visual Storytelling Tool for Non-Western Cultures25 Oct 2024
- 2024Hey GPT, Can You be More Racist? Analysis from Crowdsourced Attempts to Elicit Biased Content from Generative AI20 Oct 2024
- 2024Search Engines in an AI Era: The False Promise of Factual and Verifiable Source-Cited Responses15 Oct 2024
- 2026The Market in the Model: Latent Diffusion as Neural Economy17 Jun 2026
- 2025Position is Power: System Prompts as a Mechanism of Bias in Large Language Models (LLMs)cited by 19
- 2025The Case for "Thick Evaluations" of Cultural Representation in AIcited by 16
- 2024Search Engines in an AI Era: The False Promise of Factual and Verifiable Source-Cited Responsescited by 15
- 2025Can LLMs Express Personality Across Cultures? Introducing CulturalPersonas for Evaluating Trait Alignmentcited by 9
- 2025A Tale of Two Identities: An Ethical Audit of Human and AI-Crafted Personascited by 8
- 2024Hey GPT, Can You be More Racist? Analysis from Crowdsourced Attempts to Elicit Biased Content from Generative AIcited by 6
- 2025Invisible Filters: Cultural Bias in Hiring Evaluations Using Large Language Modelscited by 6
- 2024KAHANI: Culturally-Nuanced Visual Storytelling Tool for Non-Western Culturescited by 4
- 2025Through the Prism of Culture: Evaluating LLMs' Understanding of Indian Subcultures and Traditionscited by 3
- 2025RusCode: Russian Cultural Code Benchmark for Text-to-Image Generationcited by 1
- 2025What Is the Point of Equality in Machine Learning Fairness? Beyond Equality of Opportunitycited by 1
- 2026Prompt Revision as a Source of Cultural Bias in Text-to-Image Systemscited by 1
12 most cited of 22 All 22 (JSON)
- 2026Prompt Revision as a Source of Cultural Bias in Text-to-Image Systems10 Sep 2026
- 2026Identifying Harm in Personalized, Generative AI Systems Requires User-Centered Auditing at the Interaction Level7 Aug 2026
- 2026Harm is not Universal: Community-Specific Toxicity Detection is Urgently Needed27 Jul 2026
- 2026Mod-Guide: An LLM-based Content Moderation Feedback System to Address Insensitive Speech toward Indigenous Ethnic and Religious Minority Communities11 Jun 2026
- 2026Operationalizing Fairness in Text-to-Image Models: A Survey of Bias, Fairness Audits and Mitigation Strategies15 Apr 2026
- 2026MAPLE: Metadata Conditioned LLM Pretraining for Locale-Aware Question Answering21 Jan 2026
- 2026Responsible AI for General-Purpose Systems: Overview, Challenges, and A Path Forward19 Jan 2026
- 2025Designing and Evaluating Chain-of-Hints for Scientific Question Answering24 Oct 2025
- 2025Invisible Filters: Cultural Bias in Hiring Evaluations Using Large Language Models21 Aug 2025
- 2025NLP Meets the World: Toward Improving Conversations With the Public About Natural Language Processing Research2 Jul 2025
- 2025What Is the Point of Equality in Machine Learning Fairness? Beyond Equality of Opportunity20 Jun 2025
- 2025Can LLMs Express Personality Across Cultures? Introducing CulturalPersonas for Evaluating Trait Alignment6 Jun 2025
12 newest of 22 All 22 (JSON)