# On The Origin of Cultural Biases in Language Models: From Pre-training Data to Linguistic Phenomena

arXiv: https://arxiv.org/abs/2501.04662 · HTML: https://mlsanity.com/paper/2501.04662/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jan 2025.

## Built on by (most cited first, 22 of 22)

- [CARE: Multilingual Human Preference Learning for Cultural Awareness](https://mlsanity.com/paper/2504.05154/) (2025)
- [LLM-Based Social Simulations Require a Boundary](https://mlsanity.com/paper/2506.19806/) (2025)
- [Igniting Creative Writing in Small Language Models: LLM-as-a-Judge versus Multi-Agent Refined Rewards](https://mlsanity.com/paper/2508.21476/) (2025)
- [From RAG to Agentic RAG for Faithful Islamic Question Answering](https://mlsanity.com/paper/2601.07528/) (2026)
- [Invisible Filters: Cultural Bias in Hiring Evaluations Using Large Language Models](https://mlsanity.com/paper/2508.16673/) (2025)
- [DIWALI: Diversity and Inclusivity aWare cuLture specific Items for India: Dataset and Assessment of LLMs for Cultural Text Adaptation in Indian Context](https://mlsanity.com/paper/2509.17399/) (2025)
- [What are Foundation Models Cooking in the Post-Soviet World?](https://mlsanity.com/paper/2502.18583/) (2025)
- [Every Token Counts: Exact Likert-Scale Distributions for Measuring LLM Attitudes and Biases](https://mlsanity.com/paper/2608.10503/) (2026)
- [Camellia: Benchmarking Cultural Biases in LLMs for Asian Languages](https://mlsanity.com/paper/2510.05291/) (2025)
- [The Culture Funnel: You Can't Align What isn't in the Data](https://mlsanity.com/paper/2606.13808/) (2026)
- [On The Conceptualization and Societal Impact of Cross-Cultural Bias](https://mlsanity.com/paper/2512.21809/) (2025)
- [Can large language models interpret unstructured chat data on dynamic group decision-making processes? Evidence on joint destination choice](https://mlsanity.com/paper/2601.05582/) (2026)
- [Mitigating Cultural Bias in LLMs via Multi-Agent Cultural Debate](https://mlsanity.com/paper/2601.12091/) (2026)
- [Defining Cultural Capabilities for AI Evaluation: A Taxonomy Grounded in Intercultural Communication Theory](https://mlsanity.com/paper/2605.15990/) (2026)
- [Toward Culturally Grounded Natural Language Processing](https://mlsanity.com/paper/2603.26013/) (2026)
- [Toward Responsible and Epistemically Grounded Multilingual LLMs for Computational Social Science and Humanities](https://mlsanity.com/paper/2606.00596/) (2026)
- [Not What, But How: A Framework for Auditing LLM Responses across Positioning, Generalization, Anthropomorphism, and Maxims](https://mlsanity.com/paper/2606.02493/) (2026)
- [VakyArth: Evaluating Pragmatic Competence in LLMs across Indic Languages](https://mlsanity.com/paper/2609.01788/) (2026)
- [CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systems](https://mlsanity.com/paper/2609.22494/) (2026)
- [FineWeb-CLaR: Culture, Language, and Region Annotations for Benchmark-Aligned Corpus Auditing](https://mlsanity.com/paper/2609.25298/) (2026)
- [Evaluating Cultural Awareness of LLMs for Haitian Creole](https://mlsanity.com/paper/2609.31506/) (2026)
- [Knowing Is Not Telling: Identity Ossification in LLM Representations of Trans-border Communities](https://mlsanity.com/paper/2610.12610/) (2026)

## Abstract (arXiv)

Language Models (LMs) have been shown to exhibit a strong preference towards entities associated with Western culture when operating in non-Western languages. In this paper, we aim to uncover the origins of entity-related cultural biases in LMs by analyzing several contributing factors, including the representation of entities in pre-training data and the impact of variations in linguistic phenomena across languages. We introduce CAMeL-2, a parallel Arabic-English benchmark of 58,086 entities associated with Arab and Western cultures and 367 masked natural contexts for entities. Our evaluations using CAMeL-2 reveal reduced performance gaps between cultures by LMs when tested in English compared to Arabic. We find that LMs struggle in Arabic with entities that appear at high frequencies in pre-training, where entities can hold multiple word senses. This also extends to entities that exhibit high lexical overlap with languages that are not Arabic but use the Arabic script. Further, we show how frequency-based tokenization leads to this issue in LMs, which gets worse with larger Arabic vocabularies. We will make CAMeL-2 available at: https://github.com/tareknaous/camel2