mlsanityData to 14 Oct 2026

On The Origin of Cultural Biases in Language Models: From Pre-training Data to Linguistic Phenomena

arXiv 2501.04662 ↗First posted 8 Jan 2025cs.CLcited by 22

Not on the map. Why?

Abstract (arXiv)

Language Models (LMs) have been shown to exhibit a strong preference towards entities associated with Western culture when operating in non-Western languages. In this paper, we aim to uncover the origins of entity-related cultural biases in LMs by analyzing several contributing factors, including the representation of entities in pre-training data and the impact of variations in linguistic phenomena across languages. We introduce CAMeL-2, a parallel Arabic-English benchmark of 58,086 entities associated with Arab and Western cultures and 367 masked natural contexts for entities. Our evaluations using CAMeL-2 reveal reduced performance gaps between cultures by LMs when tested in English compared to Arabic. We find that LMs struggle in Arabic with entities that appear at high frequencies in pre-training, where entities can hold multiple word senses. This also extends to entities that exhibit high lexical overlap with languages that are not Arabic but use the Arabic script. Further, we show how frequency-based tokenization leads to this issue in LMs, which gets worse with larger Arabic vocabularies. We will make CAMeL-2 available at: https://github.com/tareknaous/camel2

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). First posted on arXiv 8 Jan 2025.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 14.

22 papers.

Sort:

By area:

  1. 2025CARE: Multilingual Human Preference Learning for Cultural Awarenesscited by 15
  2. 2025LLM-Based Social Simulations Require a Boundarycited by 12
  3. 2025Igniting Creative Writing in Small Language Models: LLM-as-a-Judge versus Multi-Agent Refined Rewardscited by 11
  4. 2026From RAG to Agentic RAG for Faithful Islamic Question Answeringcited by 9
  5. 2025Invisible Filters: Cultural Bias in Hiring Evaluations Using Large Language Modelscited by 6
  6. 2025DIWALI: Diversity and Inclusivity aWare cuLture specific Items for India: Dataset and Assessment of LLMs for Cultural Text Adaptation in Indian Contextcited by 5
  7. 2025What are Foundation Models Cooking in the Post-Soviet World?cited by 3
  8. 2026Every Token Counts: Exact Likert-Scale Distributions for Measuring LLM Attitudes and Biasescited by 3
  9. 2025Camellia: Benchmarking Cultural Biases in LLMs for Asian Languagescited by 2
  10. 2026The Culture Funnel: You Can't Align What isn't in the Datacited by 2
  11. 2025On The Conceptualization and Societal Impact of Cross-Cultural Biascited by 1
  12. 2026Can large language models interpret unstructured chat data on dynamic group decision-making processes? Evidence on joint destination choicecited by 1
10 more
  1. 2026Mitigating Cultural Bias in LLMs via Multi-Agent Cultural Debatecited by 1
  2. 2026Defining Cultural Capabilities for AI Evaluation: A Taxonomy Grounded in Intercultural Communication Theorycited by 1
  3. 2026Toward Culturally Grounded Natural Language Processingcited by 0
  4. 2026Toward Responsible and Epistemically Grounded Multilingual LLMs for Computational Social Science and Humanitiescited by 0
  5. 2026Not What, But How: A Framework for Auditing LLM Responses across Positioning, Generalization, Anthropomorphism, and Maximscited by 0
  6. 2026VakyArth: Evaluating Pragmatic Competence in LLMs across Indic Languagescited by 0
  7. 2026CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systemscited by 0
  8. 2026FineWeb-CLaR: Culture, Language, and Region Annotations for Benchmark-Aligned Corpus Auditingcited by 0
  9. 2026Evaluating Cultural Awareness of LLMs for Haitian Creolecited by 0
  10. 2026Knowing Is Not Telling: Identity Ossification in LLM Representations of Trans-border Communitiescited by 0