mlsanityData to 14 Oct 2026

TALES: A Taxonomy and Analysis of Cultural Representations in LLM-generated Stories

arXiv 2511.21322 ↗First posted 26 Nov 2025cs.HC cs.AI cs.CLcited by 10

Not on the map. Why?

Abstract (arXiv)

Millions of users across the globe turn to AI chatbots for their creative needs, inviting widespread interest in understanding how they represent diverse cultures. However, evaluating cultural representations in open-ended tasks remains challenging and underexplored. In this work, we present TALES, an evaluation of cultural misrepresentations in LLM-generated stories for diverse Indian cultural identities. First, we develop TALES-Tax, a taxonomy of cultural misrepresentations by collating insights from participants with lived experiences in India through focus groups (N=9) and individual surveys (N=15). Using TALES-Tax, we evaluate 6 models through a large-scale annotation study spanning 2925 annotations from 108 annotators with lived experience and native language proficiency from across 71 regions in India and 14 languages. Concerningly, we find that 88% of the generated stories contain misrepresentations, and such errors are more prevalent in mid- and low-resourced languages and stories based in peri-urban regions in India. We also transform the annotations into TALES-QA, a standalone question bank to evaluate the cultural knowledge of models.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 65 of its 100 references that are in the map. First posted on arXiv 26 Nov 2025.

Builds on

65 of 100 references.

References · 65

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  3. 2025Gemma 3 Technical Reportcited by 1,824
  4. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
  5. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 437
  6. 2019Measurement and Fairnesscited by 308
  7. 2024A Roadmap to Pluralistic Alignmentcited by 184
  8. 2024BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languagescited by 173
  9. 2023Art or Artifice? Large Language Models and the False Promise of Creativitycited by 172
  10. 2024Towards Measuring and Modeling "Culture" in LLMs: A Surveycited by 167
  11. 2024MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoningcited by 154
  12. 2024Investigating Cultural Alignment of Large Language Modelscited by 143
38 more
  1. 2022Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reductioncited by 138
  2. 2024Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontiercited by 129
  3. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 121
  4. 2024The Art of Saying No: Contextual Noncompliance in Language Modelscited by 105
  5. 2021Re-imagining Algorithmic Fairness in India and Beyondcited by 104
  6. 2024Survey of Cultural Awareness in Language Models: Text and Beyondcited by 103
  7. 2024CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologiescited by 98
  8. 2024Are Large Language Models Capable of Generating Human-Level Narratives?cited by 79
  9. 2024WorldValuesBench: A Large-Scale Benchmark Dataset for Multi-Cultural Value Awareness of Language Modelscited by 60
  10. 2024AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuancescited by 60
  11. 2023The Next Chapter: A Study of Large Language Models in Storytellingcited by 58
  12. 2024IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languagescited by 56
  13. 2022Re-contextualizing Fairness in NLP: The Case of Indiacited by 55
  14. 2023Typology of Risks of Generative Text-to-Image Modelscited by 55
  15. 2023AI's Regimes of Representation: A Community-centered Study of Text-to-Image Models in South Asiacited by 54
  16. 2024IndicLLMSuite: A Blueprint for Creating Pre-training and Fine-Tuning Datasets for Indian Languagescited by 52
  17. 2022How Different Groups Prioritize Ethical Values for Responsible AIcited by 49
  18. 2024Echoes in AI: Quantifying lack of plot diversity in LLM outputscited by 48
  19. 2023Inspecting the Geographical Representativeness of Images from Text-to-Image Modelscited by 44
  20. 2018A Skeleton-Based Model for Promoting Coherence Among Sentences in Narrative Story Generationcited by 42
  21. 2025Answer Matching Outperforms Multiple Choice for Language Model Evaluationcited by 41
  22. 2025Culture is Not Trivia: Sociocultural Theory for Cultural NLPcited by 40
  23. 2023When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languagescited by 32
  24. 2023Deepfakes, Phrenology, Surveillance, and More! A Taxonomy of AI Privacy Riskscited by 31
  25. 2024CulturalBench: A Robust, Diverse, and Challenging Cultural Benchmark by Human-AI CulturalTeamingcited by 30
  26. 2024Does Mapo Tofu Contain Coffee? Probing LLMs for Food-related Cultural Knowledgecited by 29
  27. 2024CulturalTeaming: AI-Assisted Interactive Red-Teaming for Challenging LLMs' (Lack of) Multicultural Knowledgecited by 28
  28. 2024Weaver: Foundation Models for Creative Writingcited by 27
  29. 2024Pron vs Prompt: Can Large Language Models already Challenge a World-Class Fiction Author at Creative Text Writing?cited by 27
  30. 2021Taxonomy of Machine Learning Safety: A Survey and Primercited by 25
  31. 2023Building Socio-culturally Inclusive Stereotype Resources with Community Engagementcited by 25
  32. 2024FoodieQA: A Multimodal Dataset for Fine-Grained Understanding of Chinese Food Culturecited by 25
  33. 2025ALFA: Aligning LLMs to Ask Good Questions A Case Study in Clinical Reasoningcited by 25
  34. 2024SeeGULL Multilingual: a Dataset of Geo-Culturally Situated Stereotypescited by 24
  35. 2024DOSA: A Dataset of Social Artifacts from Different Indian Geographical Subculturescited by 24
  36. 2024Extrinsic Evaluation of Cultural Competence in Large Language Modelscited by 24
  37. 2024How Culture Shapes What People Want From AIcited by 23
  38. 2024Evaluating Generative AI Systems is a Social Science Measurement Challengecited by 23

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 10.

10 papers.

  1. 2026LLMs Exhibit Significantly Lower Uncertainty in Creative Writing Than Professional Writerscited by 4
  2. 2026Where Do Images Come From? Analyzing Captions to Geographically Profile Datasetscited by 1
  3. 2026BavGround: A Benchmark for Regional Cultural Grounding and Dialect Competence in Bavariancited by 1
  4. 2026Sima AIunty: Caste Audit in LLM-Driven Matchmakingcited by 0
  5. 2026JuICE: A Benchmark for Evaluating LLM-Judge in Identifying Cultural Errorscited by 0
  6. 2026Characterizing Cultural Localization in AI-Generated Storiescited by 0
  7. 2026Rethinking Indic AI from a Lens of Cultural Heritage Preservationcited by 0
  8. 2026Position: AI Leaderboards Are Underserving the Global South: A Case Study from Indiacited by 0
  9. 2026Which India Survives Translation? Narrative Homogenisation Across Indian Oral Traditions in LLMscited by 0
  10. 2026Toward Cultural Alignment: Human-Centered Evaluation of Multimodal AI Stories Across Five African Communitiescited by 0