mlsanityData to 7 Oct 2026

Understanding the Capabilities and Limitations of Large Language Models for Cultural Commonsense

arXiv 2405.04655 ↗First posted 7 May 2024cs.CLcited by 50

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have demonstrated substantial commonsense understanding through numerous benchmark evaluations. However, their understanding of cultural commonsense remains largely unexamined. In this paper, we conduct a comprehensive examination of the capabilities and limitations of several state-of-the-art LLMs in the context of cultural commonsense tasks. Using several general and cultural commonsense benchmarks, we find that (1) LLMs have a significant discrepancy in performance when tested on culture-specific commonsense knowledge for different cultures; (2) LLMs' general commonsense capability is affected by cultural context; and (3) The language used to query the LLMs can impact their performance on cultural-related tasks. Our study points to the inherent bias in the cultural understanding of LLMs and provides insights that can help develop culturally aware language models.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 28 of its 41 references that are in the map. First posted on arXiv 7 May 2024.

Builds on

28 of 41 references.

References · 28

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  5. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  6. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  7. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  8. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  9. 2019Language Models as Knowledge Bases?cited by 1,922
  10. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  11. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
  12. 2023The Falcon Series of Open Language Modelscited by 526
16 more
  1. 2019COMET: Commonsense Transformers for Automatic Knowledge Graph Constructioncited by 450
  2. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 439
  3. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 424
  4. 2021Symbolic Knowledge Distillation: from General Language Models to Commonsense Modelscited by 270
  5. 2022Prompting GPT-3 To Be Reliablecited by 258
  6. 2020COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphscited by 254
  7. 2019KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoningcited by 216
  8. 2023ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learningcited by 203
  9. 2021Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Modelscited by 132
  10. 2020A Knowledge-Enhanced Pretraining Model for Commonsense Story Generationcited by 116
  11. 2020KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoningcited by 106
  12. 2022Extracting Cultural Commonsense Knowledge at Scalecited by 76
  13. 2020GenericsKB: A Knowledge Base of Generic Statementscited by 51
  14. 2022NormSAGE: Multi-Lingual Multi-Cultural Norm Discovery from Conversations On-the-Flycited by 48
  15. 2020Towards an Atlas of Cultural Commonsense for Machine Reasoningcited by 28
  16. 2023DLAMA: A Framework for Curating Culturally Diverse Facts for Probing the Knowledge of Pretrained Language Modelscited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 23; 2026: 16.

50 papers.

Sort:

By area:

  1. 2024Language Model Alignment in Multilingual Trolley Problemscited by 46
  2. 2024AraDiCE: Benchmarks for Dialectal and Cultural Capabilities in LLMscited by 46
  3. 2024CaLMQA: Exploring culturally specific long-form question answering across 23 languagescited by 31
  4. 2024NativQA: Multilingual Culturally-Aligned Natural Query for LLMscited by 30
  5. 2025Multiple LLM Agents Debate for Equitable Cultural Alignmentcited by 20
  6. 2024GraphInsight: Unlocking Insights in Large Language Models for Graph Structure Understandingcited by 18
  7. 2024Multilingual Large Language Models: A Systematic Surveycited by 18
  8. 2025CARE: Multilingual Human Preference Learning for Cultural Awarenesscited by 15
  9. 2025CULEMO: Cultural Lenses on Emotion -- Benchmarking LLMs for Cross-Cultural Emotion Understandingcited by 14
  10. 2025CulFiT: A Fine-grained Cultural-aware LLM Training Paradigm via Multilingual Critique Data Synthesiscited by 14
  11. 2025NileChat: Towards Linguistically Diverse and Culturally Aware LLMs for Local Communitiescited by 13
  12. 2025DRISHTIKON: A Multimodal Multilingual Benchmark for Testing Language Models' Understanding on Indian Culturecited by 13
38 more
  1. 2024Creating a Lens of Chinese Culture: A Multimodal Dataset for Chinese Pun Rebus Art Understandingcited by 11
  2. 2024Does Cross-Cultural Alignment Change the Commonsense Morality of Language Models?cited by 9
  3. 2024On Classification with Large Language Models in Cultural Analyticscited by 9
  4. 2024Susu Box or Piggy Bank: Assessing Cultural Commonsense Knowledge between Ghana and the U.Scited by 9
  5. 2025LLLMs: A Data-Driven Survey of Evolving Research on Limitations of Large Language Modelscited by 9
  6. 2025SaudiCulture: A Benchmark for Evaluating Large Language Models Cultural Competence within Saudi Arabiacited by 8
  7. 2025Characterizing Bias: Benchmarking Large Language Models in Simplified versus Traditional Chinesecited by 6
  8. 2025We Politely Insist: Your LLM Must Learn the Persian Art of Taarofcited by 6
  9. 2025Do Large Language Models Truly Understand Cross-cultural Differences?cited by 5
  10. 2024Chumor 2.0: Towards Benchmarking Chinese Humor Understandingcited by 4
  11. 2025Exploring Persona Sentiment Sensitivity in Personalized Dialogue Generationcited by 3
  12. 2025What are Foundation Models Cooking in the Post-Soviet World?cited by 3
  13. 2025Quality-Aware Translation Tagging in Multilingual RAG systemcited by 3
  14. 2026Common to Whom? Regional Cultural Commonsense and LLM Bias in Indiacited by 3
  15. 2025From Word to World: Evaluate and Mitigate Culture Bias in LLMs via Word Association Testcited by 2
  16. 2025Do Large Language Models Know Folktales? A Case Study of Yokai in Japanese Folktalescited by 2
  17. 2025Bias Mitigation or Cultural Commonsense? Evaluating LLMs with a Japanese Datasetcited by 2
  18. 2025Assessing Socio-Cultural Alignment and Technical Safety of Sovereign LLMscited by 2
  19. 2025Large Language Models Develop Novel Social Biases Through Adaptive Explorationcited by 2
  20. 2025Artwork Interpretation with Vision Language Models: A Case Study on Emotions and Emotion Symbolscited by 2
  21. 2025Can Language Models Handle a Non-Gregorian Calendar? The Case of the Japanese warekicited by 1
  22. 2025Everyday Physics in Korean Contexts: A Culturally Grounded Physical Reasoning Benchmarkcited by 1
  23. 2026Common Sense vs. Morality: The Curious Case of Narrative Focus Bias in LLMscited by 1
  24. 2026Distributional Open-Ended Evaluation of LLM Cultural Value Alignment Based on Value Codebookcited by 1
  25. 2026MLLM-as-a-Judge Exhibits Model Preference Biascited by 1
  26. 2026DVMap: Fine-Grained Pluralistic Value Alignment via High-Consensus Demographic-Value Mappingcited by 1
  27. 2026Jako Tako or Fluent? Presenting PoVisLE: A Polish Vision-Language Evaluationcited by 1
  28. 2025The Curious Case of Curiosity across Human Cultures and LLMscited by 0
  29. 2026To Generate or Discriminate? Methodological Considerations for Measuring Cultural Alignment in LLMscited by 0
  30. 2026Conceptual Cultural Index: A Metric for Cultural Specificity via Relative Generalitycited by 0
  31. 2026TARAZ: Persian Short-Answer Question Benchmark for Cultural Evaluation of Language Modelscited by 0
  32. 2026LiveCultureBench: a Multi-Agent, Multi-Cultural Benchmark for Large Language Models in Dynamic Social Simulationscited by 0
  33. 2026Magic, Madness, Heaven, Sin: LLM Output Diversity is Everything, Everywhere, All at Oncecited by 0
  34. 2026Culture-Aware Humorous Captioning: Multimodal Humor Generation across Cultural Contextscited by 0
  35. 2026Beyond Screenshots: Evaluating VLMs' Understanding of UI Animationscited by 0
  36. 2026Side-by-side Comparison Amplifies Dialect Bias in Language Modelscited by 0
  37. 2026CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systemscited by 0
  38. 2026CRISP: Cultural Reward Modeling for Implicit Situated Proprietycited by 0