CALM: Culturally Self-Aware Language Models
Not on the map. Why?
Abstract (arXiv)
Cultural awareness in language models is the capacity to understand and adapt to diverse cultural contexts. However, most existing approaches treat culture as static background knowledge, overlooking its dynamic and evolving nature. This limitation reduces their reliability in downstream tasks that demand genuine cultural sensitivity. In this work, we introduce CALM, a novel framework designed to endow language models with cultural self-awareness. CALM disentangles task semantics from explicit cultural concepts and latent cultural signals, shaping them into structured cultural clusters through contrastive learning. These clusters are then aligned via cross-attention to establish fine-grained interactions among related cultural features and are adaptively integrated through a Mixture-of-Experts mechanism along culture-specific dimensions. The resulting unified representation is fused with the model's original knowledge to construct a culturally grounded internal identity state, which is further enhanced through self-prompted reflective learning, enabling continual adaptation and self-correction. Extensive experiments conducted on multiple cross-cultural benchmark datasets demonstrate that CALM consistently outperforms state-of-the-art methods.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 55 of its 88 references that are in the map. First posted on arXiv 7 Jan 2026.
Builds on
55 of 88 references.
References · 55
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2023GPT-4 Technical Reportcited by 18,132
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2025Qwen3 Technical Reportcited by 8,195
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2024GPT-4o System Cardcited by 4,467
- 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
38 more
- 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
- 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
- 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
- 2025Gemma 3 Technical Reportcited by 1,686
- 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
- 2020Language-agnostic BERT Sentence Embeddingcited by 621
- 2022Mixture-of-Experts with Expert Choice Routingcited by 611
- 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 377
- 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 356
- 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 294
- 2023Orca 2: Teaching Small Language Models How to Reasoncited by 157
- 2024Investigating Cultural Alignment of Large Language Modelscited by 140
- 2023Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensionscited by 111
- 2023SeaLLMs -- Large Language Models for Southeast Asiacited by 104
- 2024Survey of Cultural Awareness in Language Models: Text and Beyondcited by 101
- 2024CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologiescited by 98
- 2023Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Dutiescited by 97
- 2023Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Promptingcited by 80
- 2023SeeGULL: A Stereotype Benchmark with Broad Geo-Cultural Coverage Leveraging Generative Modelscited by 66
- 2024Massively Multi-Cultural Knowledge Acquisition & LM Benchmarkingcited by 63
- 2022In conversation with Artificial Intelligence: aligning language models with human valuescited by 61
- 2024WorldValuesBench: A Large-Scale Benchmark Dataset for Multi-Cultural Value Awareness of Language Modelscited by 59
- 2024CULTURE-GEN: Revealing Global Cultural Perception in Language Models through Natural Language Promptingcited by 55
- 2023Exploring Cross-Cultural Differences in English Hate Speech Annotations: From Dataset Construction to Analysiscited by 44
- 2023WinoQueer: A Community-in-the-Loop Benchmark for Anti-LGBTQ+ Bias in Large Language Modelscited by 42
- 2024Can LLM Generate Culturally Relevant Commonsense QA Data? Case Study in Indonesian and Sundanesecited by 36
- 2024NativQA: Multilingual Culturally-Aligned Natural Query for LLMscited by 30
- 2024A Comprehensive Survey of Bias in LLMs: Current Landscape and Future Directionscited by 27
- 2024Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Artcited by 26
- 2024CulturePark: Boosting Cross-cultural Understanding in Large Language Modelscited by 24
- 2024Cultural Commonsense Knowledge for Intercultural Dialoguescited by 20
- 2024High-Dimension Human Value Representation in Large Language Modelscited by 20
- 2023DLAMA: A Framework for Curating Culturally Diverse Facts for Probing the Knowledge of Pretrained Language Modelscited by 17
- 2023Towards Auditing Large Language Models: Improving Text-based Stereotype Detectioncited by 12
- 2024GAMED: Knowledge Adaptive Multi-Experts Decoupling for Multimodal Fake News Detectioncited by 11
- 2024Sabi'a-2: A New Generation of Portuguese Large Language Modelscited by 10
- 2024Benchmarking Cognitive Domains for LLMs: Insights from Taiwanese Hakka Culturecited by 6
- 2024HEARTS: A Holistic Framework for Explainable, Sustainable and Robust Text Stereotype Detectioncited by 6
50 most cited of 55 All 55 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Cross-Lingual Consensus: Aligning Multilingual Cultural Knowledge via Multilingual Self-Consistency
- 2024CRAFT: Extracting and Tuning Cultural Instructions from the Wild
- 2026Aligned but Flattened: Analyzing the Trade-off between Cultural Alignment and Diversity in LLMs
- 2025Identity-Aware Large Language Models require Cultural Reasoning
- 2025Localized Cultural Knowledge is Conserved and Controllable in Large Language Models
- 2025Whispers of Many Shores: Cultural Alignment through Collaborative Cultural Expertise
- 2025CulTrace: Tracing Internal Cultural Reasoning in Large Language Models
- 2026CrossCult-KIBench: A Benchmark for Cross-Cultural Knowledge Insertion in MLLMs