mlsanityData to 14 Oct 2026

Culture is Not Trivia: Sociocultural Theory for Cultural NLP

arXiv 2502.12057 ↗First posted 17 Feb 2025cs.CL cs.CYcited by 40

Not on the map. Why?

Abstract (arXiv)

The field of cultural NLP has recently experienced rapid growth, driven by a pressing need to ensure that language technologies are effective and safe across a pluralistic user base. This work has largely progressed without a shared conception of culture, instead choosing to rely on a wide array of cultural proxies. However, this leads to a number of recurring limitations: coarse national boundaries fail to capture nuanced differences that lay within them, limited coverage restricts datasets to only a subset of usually highly-represented cultures, and a lack of dynamicity results in static cultural benchmarks that do not change as culture evolves. In this position paper, we argue that these methodological limitations are symptomatic of a theoretical gap. We draw on a well-developed theory of culture from sociocultural linguistics to fill this gap by 1) demonstrating in a case study how it can clarify methodological constraints and affordances, 2) offering theoretically-motivated paths forward to achieving cultural competence, and 3) arguing that localization is a more useful framing for the goals of much current work in cultural NLP.

Key facts. Cited by 40 papers in the map (data through 14 Oct 2026). Builds on 55 of its 109 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

55 of 109 references.

References · 55

  1. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 541
  2. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 449
  3. 2022Challenges and Strategies in Cross-Cultural NLPcited by 218
  4. 2023Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Studycited by 206
  5. 2024Towards Measuring and Modeling "Culture" in LLMs: A Surveycited by 167
  6. 2022Probing Pre-Trained Language Models for Cross-Cultural Differences in Valuescited by 149
  7. 2024Investigating Cultural Alignment of Large Language Modelscited by 143
  8. 2023Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensionscited by 112
  9. 2024CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologiescited by 98
  10. 2023Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Dutiescited by 97
  11. 2023NLPositionality: Characterizing Design Biases of Datasets and Modelscited by 94
  12. 2023SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoningcited by 94
38 more
  1. 2024KMMLU: Measuring Massive Multitask Language Understanding in Koreancited by 91
  2. 2022Extracting Cultural Commonsense Knowledge at Scalecited by 76
  3. 2023KoBBQ: Korean Bias Benchmark for Question Answeringcited by 71
  4. 2024MAPO: Advancing Multilingual Reasoning through Multilingual Alignment-as-Preference Optimizationcited by 69
  5. 2023SeeGULL: A Stereotype Benchmark with Broad Geo-Cultural Coverage Leveraging Generative Modelscited by 66
  6. 2024CLIcK: A Benchmark Dataset of Cultural and Linguistic Intelligence in Koreancited by 66
  7. 2024Massively Multi-Cultural Knowledge Acquisition & LM Benchmarkingcited by 63
  8. 2023NormBank: A Knowledge Bank of Situational Social Normscited by 60
  9. 2024WorldValuesBench: A Large-Scale Benchmark Dataset for Multi-Cultural Value Awareness of Language Modelscited by 60
  10. 2024CULTURE-GEN: Revealing Global Cultural Perception in Language Models through Natural Language Promptingcited by 55
  11. 2023Multilingual Language Models are not Multicultural: A Case Study in Emotioncited by 54
  12. 2022Multi-VALUE: A Framework for Cross-Dialectal English NLPcited by 53
  13. 2023Benchmarking Machine Translation with Cultural Awarenesscited by 53
  14. 2022NormSAGE: Multi-Lingual Multi-Cultural Norm Discovery from Conversations On-the-Flycited by 48
  15. 2023Multi-lingual and Multi-cultural Figurative Language Understandingcited by 47
  16. 2024Evaluating Cultural and Social Awareness of LLM Web Agentscited by 45
  17. 2024IndoCulture: Exploring Geographically-Influenced Cultural Commonsense Reasoning Across Eleven Indonesian Provincescited by 39
  18. 2024An image speaks a thousand words, but can everyone listen? On image transcreation for cultural relevancecited by 36
  19. 2023EtiCor: Corpus for Analyzing LLMs for Etiquettescited by 33
  20. 2024KorNAT: LLM Alignment Benchmark for Korean Social Values and Common Knowledgecited by 33
  21. 2024Does Mapo Tofu Contain Coffee? Probing LLMs for Food-related Cultural Knowledgecited by 29
  22. 2024ViSAGe: A Global-Scale Analysis of Visual Stereotypes in Text-to-Image Generationcited by 27
  23. 2023Building Socio-culturally Inclusive Stereotype Resources with Community Engagementcited by 25
  24. 2024SeeGULL Multilingual: a Dataset of Geo-Culturally Situated Stereotypescited by 24
  25. 2024DOSA: A Dataset of Social Artifacts from Different Indian Geographical Subculturescited by 24
  26. 2024Extrinsic Evaluation of Cultural Competence in Large Language Modelscited by 24
  27. 2023Sociocultural Norm Similarities and Differences via Situational Alignment and Explainable Textual Entailmentcited by 22
  28. 2023Cross-Cultural Transfer Learning for Chinese Offensive Language Detectioncited by 21
  29. 2024What Do Dialect Speakers Want? A Survey of Attitudes Towards Language Technology for German Dialectscited by 21
  30. 2021Characterizing English Variation across Social Media Communities with BERTcited by 20
  31. 2022Evaluating the Diversity, Equity and Inclusion of NLP Technology: A Case Study for Indian Languagescited by 20
  32. 2023Modeling Cross-Cultural Pragmatic Inference with Codenames Duetcited by 20
  33. 2017Community Identity and User Engagement in a Multi-Community Landscapecited by 18
  34. 2023Cultural Compass: Predicting Transfer Learning Success in Offensive Language Detection with Cultural Featurescited by 18
  35. 2024Bridging Cultural Nuances in Dialogue Agents through Cultural Value Surveyscited by 18
  36. 2023Geographical Erasure in Language Generationcited by 15
  37. 2024RENOVI: A Benchmark Towards Remediating Norm Violations in Socio-Cultural Conversationscited by 14
  38. 2023GD-COMET: A Geo-Diverse Commonsense Inference Modelcited by 13

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 21; 2026: 19.

40 papers.

Sort:

By area:

  1. 2025ELEPHANT: Measuring and understanding social sycophancy in LLMscited by 129
  2. 2025Towards Trustworthy GUI Agents: A Surveycited by 34
  3. 2025Hire Your Anthropologist! Rethinking Culture Benchmarks Through an Anthropological Lenscited by 14
  4. 2025Provocations from the Humanities for Generative AI Researchcited by 10
  5. 2025Meaning Is Not A Metric: Using LLMs to make cultural context legible at scalecited by 10
  6. 2025Culture is Everywhere: A Call for Intentionally Cultural Evaluationcited by 10
  7. 2025TALES: A Taxonomy and Analysis of Cultural Representations in LLM-generated Storiescited by 10
  8. 2026Computational Hermeneutics: Evaluating generative AI as a cultural technologycited by 8
  9. 2025Do Large Language Models Truly Understand Cross-cultural Differences?cited by 6
  10. 2025Is It Bad to Work All the Time? Cross-Cultural Evaluation of Social Norm Biases in GPT-4cited by 5
  11. 2025FanarGuard: A Culturally-Aware Moderation Filter for Arabic Language Modelscited by 5
  12. 2025Identity-Aware Large Language Models require Cultural Reasoningcited by 4
28 more
  1. 2026Steering LLMs for Culturally Localized Generationcited by 4
  2. 2025Evaluation of Cultural Competence of Vision-Language Modelscited by 3
  3. 2025Research Borderlands: Analysing Writing Across Research Culturescited by 3
  4. 2025MELAC: Massive Evaluation of Large Language Models with Alignment of Culture in Persian Languagecited by 3
  5. 2025From Word to World: Evaluate and Mitigate Culture Bias in LLMs via Word Association Testcited by 2
  6. 2025What is a protest anyway? Codebook conceptualization is still a first-order concern in LLM-era classificationcited by 2
  7. 2025Culture Cartography: Mapping the Landscape of Cultural Knowledgecited by 2
  8. 2026Responsible Evaluation of AI for Mental Healthcited by 2
  9. 2025Exploring Cultural Nuances in Emotion Perception Across 15 African Languagescited by 1
  10. 2025CAIRe: Cultural Attribution of Images by Retrieval-Augmented Evaluationcited by 1
  11. 2026No Shortcuts to Culture: Indonesian Multi-hop Question Answering for Complex Cultural Understandingcited by 1
  12. 2026BanglaSocialBench: A Benchmark for Evaluating Sociopragmatic and Cultural Alignment of LLMs in Bangladeshi Social Interactioncited by 1
  13. 2026Defining Cultural Capabilities for AI Evaluation: A Taxonomy Grounded in Intercultural Communication Theorycited by 1
  14. 2026BavGround: A Benchmark for Regional Cultural Grounding and Dialect Competence in Bavariancited by 1
  15. 2025Against 'softmaxing' culturecited by 0
  16. 2025Debiasing Large Language Models in Thai Political Stance Detection via Counterfactual Calibrationcited by 0
  17. 2026To Generate or Discriminate? Methodological Considerations for Measuring Cultural Alignment in LLMscited by 0
  18. 2026Toward Culturally Grounded Natural Language Processingcited by 0
  19. 2026Cross-Cultural Value Attribution in Large Vision-Language Modelscited by 0
  20. 2026SemEval-2026 Task 7: Everyday Knowledge Across Diverse Languages and Culturescited by 0
  21. 2026JuICE: A Benchmark for Evaluating LLM-Judge in Identifying Cultural Errorscited by 0
  22. 2026Which Institutional Frameworks Do Chatbots Assume? Auditing Jurisdictional Defaults in Multilingual LLMscited by 0
  23. 2026Language Models as Measurement Apparatus for Culturecited by 0
  24. 2026CCBENCH: Assessing LLM Cultural Competence via Implicitly Signaled Norms using Health Queriescited by 0
  25. 2026When Trivia Is Not Trivial: Everyday Knowledge Failures in Multilingual LLMscited by 0
  26. 2026What Limits Us? Analyzing Self-Reported Limitations in NLP Researchcited by 0
  27. 2026CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systemscited by 0
  28. 2026Supporting and Performing Culture from the Insidecited by 0