mlsanityData to 7 Oct 2026

Survey of Cultural Awareness in Language Models: Text and Beyond

arXiv 2411.00860 ↗First posted 30 Oct 2024cs.CL cs.CVcited by 101

Not on the map. Why?

Abstract (arXiv)

Large-scale deployment of large language models (LLMs) in various applications, such as chatbots and virtual assistants, requires LLMs to be culturally sensitive to the user to ensure inclusivity. Culture has been widely studied in psychology and anthropology, and there has been a recent surge in research on making LLMs more culturally inclusive in LLMs that goes beyond multilinguality and builds on findings from psychology and anthropology. In this paper, we survey efforts towards incorporating cultural awareness into text-based and multimodal LLMs. We start by defining cultural awareness in LLMs, taking the definitions of culture from anthropology and psychology as a point of departure. We then examine methodologies adopted for creating cross-cultural datasets, strategies for cultural inclusion in downstream tasks, and methodologies that have been used for benchmarking cultural awareness in LLMs. Further, we discuss the ethical implications of cultural alignment, the role of Human-Computer Interaction in driving cultural inclusion in LLMs, and the role of cultural alignment in driving social science research. We finally provide pointers to future research based on our findings about gaps in the literature.

Key facts. Cited by 101 papers in the map (data through 7 Oct 2026). Builds on 142 of its 478 references that are in the map. First posted on arXiv 30 Oct 2024.

Builds on

142 of 478 references.

References · 142

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  3. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  4. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  5. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 424
  6. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
  7. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  8. 2023Does Writing with Language Models Reduce Content Diversity?cited by 190
  9. 2024BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languagescited by 171
  10. 2024Towards Measuring and Modeling "Culture" in LLMs: A Surveycited by 166
  11. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 154
  12. 2023Having Beer after Prayer? Measuring Cultural Bias in Large Language Modelscited by 153
38 more
  1. 2024Large language models that replace human participants can harmfully misportray and flatten identity groupscited by 137
  2. 2024CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmarkcited by 118
  3. 2023Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensionscited by 111
  4. 2024Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaborationcited by 111
  5. 2022The Ghost in the Machine has an American accent: value conflict in GPT-3cited by 105
  6. 2023SeaLLMs -- Large Language Models for Southeast Asiacited by 104
  7. 2024Benchmarking Vision Language Models for Cultural Understandingcited by 100
  8. 2024CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologiescited by 98
  9. 2024MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answeringcited by 93
  10. 2024KMMLU: Measuring Massive Multitask Language Understanding in Koreancited by 91
  11. 2023A Bi-Step Grounding Paradigm for Large Language Models in Recommendation Systemscited by 73
  12. 2023Humanoid Agents: Platform for Simulating Human-like Generative Agentscited by 73
  13. 2023Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Modelscited by 68
  14. 2024Massively Multi-Cultural Knowledge Acquisition & LM Benchmarkingcited by 63
  15. 2023Automating Customer Service using LangChain: Building custom open-source GPT Chatbot for organizationscited by 59
  16. 2024CULTURE-GEN: Revealing Global Cultural Perception in Language Models through Natural Language Promptingcited by 55
  17. 2023Benchmarking Machine Translation with Cultural Awarenesscited by 53
  18. 2020GenericsKB: A Knowledge Base of Generic Statementscited by 51
  19. 2024CultureLLM: Incorporating Cultural Differences into Large Language Modelscited by 49
  20. 2024DIALECTBENCH: A NLP Benchmark for Dialects, Varieties, and Closely-Related Languagescited by 49
  21. 2024(Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Textscited by 49
  22. 2024NormAd: A Framework for Measuring the Cultural Adaptability of Large Language Modelscited by 47
  23. 2024Beyond Preferences in AI Alignmentcited by 46
  24. 2024AraDiCE: Benchmarks for Dialectal and Cultural Capabilities in LLMscited by 46
  25. 2024COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuningcited by 45
  26. 2024MuChoMusic: Evaluating Music Understanding in Multimodal Audio-Language Modelscited by 43
  27. 2024Should We Respect LLMs? A Cross-Lingual Study on the Influence of Prompt Politeness on LLM Performancecited by 41
  28. 2024SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languagescited by 41
  29. 2024IndicVoices: Towards building an Inclusive Multilingual Speech Dataset for Indian Languagescited by 39
  30. 2024IndoCulture: Exploring Geographically-Influenced Cultural Commonsense Reasoning Across Eleven Indonesian Provincescited by 38
  31. 2021M6-UFC: Unifying Multi-Modal Controls for Conditional Image Synthesis via Non-Autoregressive Generative Transformerscited by 36
  32. 2024Can LLM Generate Culturally Relevant Commonsense QA Data? Case Study in Indonesian and Sundanesecited by 36
  33. 2024An image speaks a thousand words, but can everyone listen? On image transcreation for cultural relevancecited by 36
  34. 2024RTP-LX: Can LLMs Evaluate Toxicity in Multilingual Scenarios?cited by 34
  35. 2024KorNAT: LLM Alignment Benchmark for Korean Social Values and Common Knowledgecited by 33
  36. 2024MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMscited by 32
  37. 2024CaLMQA: Exploring culturally specific long-form question answering across 23 languagescited by 31
  38. 2024TurkishMMLU: Measuring Massive Multitask Language Understanding in Turkishcited by 31

50 most cited of 142 All 142 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 54; 2026: 46.

101 papers.

Sort:

By area:

  1. 2026Lost in Simulation: LLM-Simulated Users are Unreliable Proxies for Human Users in Agentic Evaluationscited by 36
  2. 2025LLM-based Human Simulations Have Not Yet Been Reliablecited by 30
  3. 2025Show Me the Work: Fact-Checkers' Requirements for Explainable Automated Fact-Checkingcited by 29
  4. 2025Presumed Cultural Identity: How Names Shape LLM Responsescited by 21
  5. 2025PalmX 2025: The First Shared Task on Benchmarking LLMs on Arabic and Islamic Culturecited by 20
  6. 2025The Case for "Thick Evaluations" of Cultural Representation in AIcited by 16
  7. 2025CARE: Multilingual Human Preference Learning for Cultural Awarenesscited by 15
  8. 2025CulFiT: A Fine-grained Cultural-aware LLM Training Paradigm via Multilingual Critique Data Synthesiscited by 14
  9. 2025MAKIEval: A Multilingual Automatic WiKidata-based Framework for Cultural Awareness Evaluation for LLMscited by 13
  10. 2025DRISHTIKON: A Multimodal Multilingual Benchmark for Testing Language Models' Understanding on Indian Culturecited by 13
  11. 2025Hire Your Anthropologist! Rethinking Culture Benchmarks Through an Anthropological Lenscited by 13
  12. 2025Pairwise Calibrated Rewards for Pluralistic Alignmentcited by 12
38 more
  1. 2025Igniting Creative Writing in Small Language Models: LLM-as-a-Judge versus Multi-Agent Refined Rewardscited by 11
  2. 2025Culture is Everywhere: A Call for Intentionally Cultural Evaluationcited by 10
  3. 2025TALES: A Taxonomy and Analysis of Cultural Representations in LLM-generated Storiescited by 10
  4. 2025Provocations from the Humanities for Generative AI Researchcited by 8
  5. 2025When Cultures Meet: Multicultural Text-to-Image Generationcited by 7
  6. 2025MoVa: Towards Generalizable Classification of Human Morals and Valuescited by 7
  7. 2025Pearl: A Multimodal Culturally-Aware Arabic Instruction Datasetcited by 6
  8. 2025CultureSynth: A Hierarchical Taxonomy-Guided and Retrieval-Augmented Framework for Cultural Question-Answer Synthesiscited by 6
  9. 2025Towards Geo-Culturally Grounded LLM Generationscited by 5
  10. 2025LLM-C3MOD: A Human-LLM Collaborative System for Cross-Cultural Hate Speech Moderationcited by 5
  11. 2025Hanfu-Bench: A Multimodal Benchmark on Cross-Temporal Cultural Understanding and Transcreationcited by 5
  12. 2025Cross-Lingual Transfer of Cultural Knowledge: An Asymmetric Phenomenoncited by 5
  13. 2025Culturally-Aware Conversations: A Framework & Benchmark for LLMscited by 5
  14. 2024Chumor 2.0: Towards Benchmarking Chinese Humor Understandingcited by 4
  15. 2025Cultural Alignment in Large Language Models Using Soft Prompt Tuningcited by 4
  16. 2025A large-scale evaluation of commonsense knowledge in humans and large language modelscited by 4
  17. 2025IRLBench: A Multi-modal, Culturally Grounded, Parallel Irish-English Benchmark for Open-Ended LLM Reasoning Evaluationcited by 4
  18. 2025UPDESH: Synthesizing Grounded Instruction Tuning Data for 13 Indic Languagescited by 4
  19. 2025Bias in, Bias out: Annotation Bias in Multilingual Large Language Modelscited by 4
  20. 2026Auditing Preferences for Brands and Cultures in LLMscited by 4
  21. 2025BenchHub: A Unified Benchmark Suite for Holistic and Customizable LLM Evaluationcited by 3
  22. 2025Research Borderlands: Analysing Writing Across Research Culturescited by 3
  23. 2025MCEval: A Dynamic Framework for Fair Multilingual Cultural Evaluation of LLMscited by 3
  24. 2025MELAC: Massive Evaluation of Large Language Models with Alignment of Culture in Persian Languagecited by 3
  25. 2025BLEnD-Vis: Benchmarking Multimodal Cultural Understanding in Vision Language Modelscited by 3
  26. 2025Beyond MCQ: An Open-Ended Arabic Cultural QA Benchmark with Dialect Variantscited by 3
  27. 2025Finding Culture-Sensitive Neurons in Vision-Language Modelscited by 3
  28. 2025World in a Frame: Understanding Culture Mixing as a New Challenge for Vision-Language Modelscited by 3
  29. 2026On the Wings of Imagination: Conflicting Script-based Multi-role Framework for Humor Caption Generationcited by 3
  30. 2026Mind the Gap in Cultural Alignment: Task-Aware Culture Management for Large Language Modelscited by 3
  31. 2026Many Dialects, Many Languages, One Cultural Lens: Evaluating Multilingual VLMs for Bengali Culture Understanding Across Historically Linked Languages and Regional Dialectscited by 3
  32. 2025A Framework for Situating Innovations, Opportunities, and Challenges in Advancing Vertical Systems with Large AI Modelscited by 2
  33. 2025An evaluation of LLMs for generating movie reviews: GPT-4o, Gemini-2.0 and DeepSeek-V3cited by 2
  34. 2025CultranAI at PalmX 2025: Data Augmentation for Cultural Knowledge Representationcited by 2
  35. 2025Camellia: Benchmarking Cultural Biases in LLMs for Asian Languagescited by 2
  36. 2026CALM: Culturally Self-Aware Language Modelscited by 2
  37. 2026Cultural Counterfactuals: Evaluating Cultural Biases in Large Vision-Language Models with Counterfactual Examplescited by 2
  38. 2026Cultural Perspectives and Expectations for Generative AI: A Global Survey Approachcited by 2

50 most cited of 101 All 101 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025The American Ghost in the Machine: How language models align culturally and the effects of cultural prompting
  2. 2024Evaluating Cultural Awareness of LLMs for Yoruba, Malayalam, and English
  3. 2025'Too much alignment; not enough culture': Re-balancing cultural alignment practices in LLMs
  4. 2024Investigating Cultural Alignment of Large Language Models
  5. 2025DeepSeek's WEIRD Behavior: The cultural alignment of Large Language Models and the effects of prompt language and cultural prompting
  6. 2025Randomness, Not Representation: The Unreliability of Evaluating Cultural Alignment in LLMs
  7. 2025From Word to World: Evaluate and Mitigate Culture Bias in LLMs via Word Association Test
  8. 2025Do Large Language Models Truly Understand Cross-cultural Differences?