XCR-Bench: Benchmarking Cross-Cultural Reasoning in LLMs via Culture-Specific Items and Hall's Triad
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
Cross-cultural competence in large language models (LLMs) requires understanding and adapting Culture-Specific Items (CSIs) across varying cultural contexts. However, progress in evaluating this capability remains limited by the lack of high-quality CSI-annotated corpora with parallel cross-cultural sentence pairs. We introduce XCR-Bench, a Cross(X)-Cultural Reasoning Benchmark containing 4.1k parallel sentences and 1,098 CSIs across three reasoning tasks. XCR-Bench integrates Newmark's CSI framework with Hall's Triad of Culture, enabling evaluation across levels of cultural visibility -- from observable practices to implicit social norms and values. Experiments on eight multilingual LLMs show that state-of-the-art models exhibit consistent weaknesses in identifying and adapting specific categories of CSIs, revealing a gap between surface-level recall and explicit cultural reasoning. Performance declines significantly on culturally sensitive categories and deeper cultural levels (p<0.005, 8/8 models), and adaptation quality varies systematically across target cultures and Bengali regional variants, indicating encoded regional and ethno-religious biases even within a single linguistic setting. We publicly release the corpus and code to support future research on cross-cultural NLP.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 17 of its 48 references that are in the map. First posted on arXiv 20 Jan 2026.
Builds on
17 of 48 references.
References · 17
- 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 437
- 2022Challenges and Strategies in Cross-Cultural NLPcited by 218
- 2024Towards Measuring and Modeling "Culture" in LLMs: A Surveycited by 167
- 2023ExpertPrompting: Instructing Large Language Models to be Distinguished Expertscited by 142
- 2022The Ghost in the Machine has an American accent: value conflict in GPT-3cited by 105
- 2024WorldValuesBench: A Large-Scale Benchmark Dataset for Multi-Cultural Value Awareness of Language Modelscited by 60
- 2024How Well Do LLMs Represent Values Across Cultures? Empirical Analysis of LLM Responses Based on Hofstede Cultural Dimensionscited by 52
- 2024NormAd: A Framework for Measuring the Cultural Adaptability of Large Language Modelscited by 48
- 2024Translating Across Cultures: LLMs for Intralingual Cultural Adaptationcited by 16
- 2019Pre-Training with Whole Word Masking for Chinese BERTcited by 14
- 2020The ApposCorpus: A new multilingual, multi-domain dataset for factual appositive generationcited by 6
- 2024Pre-Trained Language Models Represent Some Geographic Populations Better Than Otherscited by 6
5 more
- 2025Do Large Language Models Truly Understand Cross-cultural Differences?cited by 6
- 2025When Tom Eats Kimchi: Evaluating Cultural Bias of Multimodal Large Language Models in Cultural Mixture Contextscited by 3
- 2025Culture In a Frame: C3B as a Comic-Based Benchmark for Multimodal Culturally Awarenesscited by 2
- 2025Semantic Label Drift in Cross-Cultural Translationcited by 2
- 2025From n-gram to Attention: How Model Architectures Learn and Propagate Bias in Language Modelingcited by 1
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026CultureForest: Understanding and Evaluating Cultural Norm Grounded Reasoning in LLMscited by 1
- 2026SEA-NLI: Natural Language Inference as a Lens into Southeast Asian Cultural Understandingcited by 0
- 2026Large-scale factor analysis shows machine intelligence is only partially interpretablecited by 0
- 2026Large-scale factor analysis shows machine intelligence is only partially interpretable29 Sep 2026
- 2026SEA-NLI: Natural Language Inference as a Lens into Southeast Asian Cultural Understanding2 Jun 2026
- 2026CultureForest: Understanding and Evaluating Cultural Norm Grounded Reasoning in LLMs1 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2025CRaFT: An Explanation-Based Framework for Evaluating Cultural Reasoning in Multilingual Language Models
- 2025CURE: Cultural Understanding and Reasoning Evaluation - A Framework for "Thick" Culture Alignment Evaluation in LLMs
- 2025Cross-Cultural Transfer of Commonsense Reasoning in LLMs: Evidence from the Arab World
- 2025CulFiT: A Fine-grained Cultural-aware LLM Training Paradigm via Multilingual Critique Data Synthesis
- 2025From Word to World: Evaluate and Mitigate Culture Bias in LLMs via Word Association Test
- 2026Cross-Lingual Consensus: Aligning Multilingual Cultural Knowledge via Multilingual Self-Consistency
- 2026CrossCult-KIBench: A Benchmark for Cross-Cultural Knowledge Insertion in MLLMs
- 2025MMA-ASIA: A Multilingual and Multimodal Alignment Framework for Culturally-Grounded Evaluation