mlsanityData to 7 Oct 2026

CALM: Culturally Self-Aware Language Models

arXiv 2601.03483 ↗First posted 7 Jan 2026cs.CL cs.CY cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Cultural awareness in language models is the capacity to understand and adapt to diverse cultural contexts. However, most existing approaches treat culture as static background knowledge, overlooking its dynamic and evolving nature. This limitation reduces their reliability in downstream tasks that demand genuine cultural sensitivity. In this work, we introduce CALM, a novel framework designed to endow language models with cultural self-awareness. CALM disentangles task semantics from explicit cultural concepts and latent cultural signals, shaping them into structured cultural clusters through contrastive learning. These clusters are then aligned via cross-attention to establish fine-grained interactions among related cultural features and are adaptively integrated through a Mixture-of-Experts mechanism along culture-specific dimensions. The resulting unified representation is fused with the model's original knowledge to construct a culturally grounded internal identity state, which is further enhanced through self-prompted reflective learning, enabling continual adaptation and self-correction. Extensive experiments conducted on multiple cross-cultural benchmark datasets demonstrate that CALM consistently outperforms state-of-the-art methods.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 55 of its 88 references that are in the map. First posted on arXiv 7 Jan 2026.

Builds on

55 of 88 references.

References · 55

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  7. 2025Qwen3 Technical Reportcited by 8,195
  8. 2020Scaling Laws for Neural Language Modelscited by 6,411
  9. 2024GPT-4o System Cardcited by 4,467
  10. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  11. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  12. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
38 more
  1. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  2. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  3. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  4. 2025Gemma 3 Technical Reportcited by 1,686
  5. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  6. 2020Language-agnostic BERT Sentence Embeddingcited by 621
  7. 2022Mixture-of-Experts with Expert Choice Routingcited by 611
  8. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 377
  9. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 356
  10. 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 294
  11. 2023Orca 2: Teaching Small Language Models How to Reasoncited by 157
  12. 2024Investigating Cultural Alignment of Large Language Modelscited by 140
  13. 2023Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensionscited by 111
  14. 2023SeaLLMs -- Large Language Models for Southeast Asiacited by 104
  15. 2024Survey of Cultural Awareness in Language Models: Text and Beyondcited by 101
  16. 2024CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologiescited by 98
  17. 2023Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Dutiescited by 97
  18. 2023Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Promptingcited by 80
  19. 2023SeeGULL: A Stereotype Benchmark with Broad Geo-Cultural Coverage Leveraging Generative Modelscited by 66
  20. 2024Massively Multi-Cultural Knowledge Acquisition & LM Benchmarkingcited by 63
  21. 2022In conversation with Artificial Intelligence: aligning language models with human valuescited by 61
  22. 2024WorldValuesBench: A Large-Scale Benchmark Dataset for Multi-Cultural Value Awareness of Language Modelscited by 59
  23. 2024CULTURE-GEN: Revealing Global Cultural Perception in Language Models through Natural Language Promptingcited by 55
  24. 2023Exploring Cross-Cultural Differences in English Hate Speech Annotations: From Dataset Construction to Analysiscited by 44
  25. 2023WinoQueer: A Community-in-the-Loop Benchmark for Anti-LGBTQ+ Bias in Large Language Modelscited by 42
  26. 2024Can LLM Generate Culturally Relevant Commonsense QA Data? Case Study in Indonesian and Sundanesecited by 36
  27. 2024NativQA: Multilingual Culturally-Aligned Natural Query for LLMscited by 30
  28. 2024A Comprehensive Survey of Bias in LLMs: Current Landscape and Future Directionscited by 27
  29. 2024Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Artcited by 26
  30. 2024CulturePark: Boosting Cross-cultural Understanding in Large Language Modelscited by 24
  31. 2024Cultural Commonsense Knowledge for Intercultural Dialoguescited by 20
  32. 2024High-Dimension Human Value Representation in Large Language Modelscited by 20
  33. 2023DLAMA: A Framework for Curating Culturally Diverse Facts for Probing the Knowledge of Pretrained Language Modelscited by 17
  34. 2023Towards Auditing Large Language Models: Improving Text-based Stereotype Detectioncited by 12
  35. 2024GAMED: Knowledge Adaptive Multi-Experts Decoupling for Multimodal Fake News Detectioncited by 11
  36. 2024Sabi'a-2: A New Generation of Portuguese Large Language Modelscited by 10
  37. 2024Benchmarking Cognitive Domains for LLMs: Insights from Taiwanese Hakka Culturecited by 6
  38. 2024HEARTS: A Holistic Framework for Explainable, Sustainable and Robust Text Stereotype Detectioncited by 6

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Cross-Lingual Consensus: Aligning Multilingual Cultural Knowledge via Multilingual Self-Consistency
  2. 2024CRAFT: Extracting and Tuning Cultural Instructions from the Wild
  3. 2026Aligned but Flattened: Analyzing the Trade-off between Cultural Alignment and Diversity in LLMs
  4. 2025Identity-Aware Large Language Models require Cultural Reasoning
  5. 2025Localized Cultural Knowledge is Conserved and Controllable in Large Language Models
  6. 2025Whispers of Many Shores: Cultural Alignment through Collaborative Cultural Expertise
  7. 2025CulTrace: Tracing Internal Cultural Reasoning in Large Language Models
  8. 2026CrossCult-KIBench: A Benchmark for Cross-Cultural Knowledge Insertion in MLLMs