# XCR-Bench: Benchmarking Cross-Cultural Reasoning in LLMs via Culture-Specific Items and Hall's Triad

arXiv: https://arxiv.org/abs/2601.14063 · HTML: https://mlsanity.com/paper/2601.14063/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 17 of its 48 references that are in the map.
- First posted on arXiv 20 Jan 2026.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 17 of 17)

- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [Challenges and Strategies in Cross-Cultural NLP](https://mlsanity.com/paper/2203.10020/) (2022)
- [Towards Measuring and Modeling "Culture" in LLMs: A Survey](https://mlsanity.com/paper/2403.15412/) (2024)
- [ExpertPrompting: Instructing Large Language Models to be Distinguished Experts](https://mlsanity.com/paper/2305.14688/) (2023)
- [The Ghost in the Machine has an American accent: value conflict in GPT-3](https://mlsanity.com/paper/2203.07785/) (2022)
- [WorldValuesBench: A Large-Scale Benchmark Dataset for Multi-Cultural Value Awareness of Language Models](https://mlsanity.com/paper/2404.16308/) (2024)
- [How Well Do LLMs Represent Values Across Cultures? Empirical Analysis of LLM Responses Based on Hofstede Cultural Dimensions](https://mlsanity.com/paper/2406.14805/) (2024)
- [NormAd: A Framework for Measuring the Cultural Adaptability of Large Language Models](https://mlsanity.com/paper/2404.12464/) (2024)
- [Translating Across Cultures: LLMs for Intralingual Cultural Adaptation](https://mlsanity.com/paper/2406.14504/) (2024)
- [Pre-Training with Whole Word Masking for Chinese BERT](https://mlsanity.com/paper/1906.08101/) (2019)
- [The ApposCorpus: A new multilingual, multi-domain dataset for factual appositive generation](https://mlsanity.com/paper/2011.03287/) (2020)
- [Pre-Trained Language Models Represent Some Geographic Populations Better Than Others](https://mlsanity.com/paper/2403.11025/) (2024)
- [Do Large Language Models Truly Understand Cross-cultural Differences?](https://mlsanity.com/paper/2512.07075/) (2025)
- [When Tom Eats Kimchi: Evaluating Cultural Bias of Multimodal Large Language Models in Cultural Mixture Contexts](https://mlsanity.com/paper/2503.16826/) (2025)
- [Culture In a Frame: C3B as a Comic-Based Benchmark for Multimodal Culturally Awareness](https://mlsanity.com/paper/2510.00041/) (2025)
- [Semantic Label Drift in Cross-Cultural Translation](https://mlsanity.com/paper/2510.25967/) (2025)
- [From n-gram to Attention: How Model Architectures Learn and Propagate Bias in Language Modeling](https://mlsanity.com/paper/2505.12381/) (2025)

## Built on by (most cited first, 3 of 3)

- [CultureForest: Understanding and Evaluating Cultural Norm Grounded Reasoning in LLMs](https://mlsanity.com/paper/2606.01879/) (2026)
- [SEA-NLI: Natural Language Inference as a Lens into Southeast Asian Cultural Understanding](https://mlsanity.com/paper/2606.03284/) (2026)
- [Large-scale factor analysis shows machine intelligence is only partially interpretable](https://mlsanity.com/paper/2609.36515/) (2026)

## Abstract (arXiv)

Cross-cultural competence in large language models (LLMs) requires understanding and adapting Culture-Specific Items (CSIs) across varying cultural contexts. However, progress in evaluating this capability remains limited by the lack of high-quality CSI-annotated corpora with parallel cross-cultural sentence pairs. We introduce XCR-Bench, a Cross(X)-Cultural Reasoning Benchmark containing 4.1k parallel sentences and 1,098 CSIs across three reasoning tasks. XCR-Bench integrates Newmark's CSI framework with Hall's Triad of Culture, enabling evaluation across levels of cultural visibility -- from observable practices to implicit social norms and values. Experiments on eight multilingual LLMs show that state-of-the-art models exhibit consistent weaknesses in identifying and adapting specific categories of CSIs, revealing a gap between surface-level recall and explicit cultural reasoning. Performance declines significantly on culturally sensitive categories and deeper cultural levels (p<0.005, 8/8 models), and adaptation quality varies systematically across target cultures and Bengali regional variants, indicating encoded regional and ethno-religious biases even within a single linguistic setting. We publicly release the corpus and code to support future research on cross-cultural NLP.