# CultureScope: A Dimensional Lens for Probing Cultural Understanding in LLMs

arXiv: https://arxiv.org/abs/2509.16188 · HTML: https://mlsanity.com/paper/2509.16188/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 31 references that are in the map.
- First posted on arXiv 19 Sep 2025.

## Builds on (most cited first, 13 of 13)

- [ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning](https://mlsanity.com/paper/2304.05613/) (2023)
- [Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation](https://mlsanity.com/paper/2412.03304/) (2024)
- [BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languages](https://mlsanity.com/paper/2406.09948/) (2024)
- [Benchmarking Vision Language Models for Cultural Understanding](https://mlsanity.com/paper/2407.10920/) (2024)
- [SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoning](https://mlsanity.com/paper/2309.04766/) (2023)
- [A Survey on Multilingual Large Language Models: Corpora, Alignment, and Bias](https://mlsanity.com/paper/2404.00929/) (2024)
- [AraDiCE: Benchmarks for Dialectal and Cultural Capabilities in LLMs](https://mlsanity.com/paper/2409.11404/) (2024)
- [Assessing LLMs for Moral Value Pluralism](https://mlsanity.com/paper/2312.10075/) (2023)
- [Can Language Models Reason about Individualistic Human Values and Preferences?](https://mlsanity.com/paper/2410.03868/) (2024)
- [KorNAT: LLM Alignment Benchmark for Korean Social Values and Common Knowledge](https://mlsanity.com/paper/2402.13605/) (2024)
- [LLM-GLOBE: A Benchmark Evaluating the Cultural Values Embedded in LLM Output](https://mlsanity.com/paper/2411.06032/) (2024)
- [JMMMU: A Japanese Massive Multi-discipline Multimodal Understanding Benchmark for Culture-aware Evaluation](https://mlsanity.com/paper/2410.17250/) (2024)
- [Multilingual Pretraining Using a Large Corpus Machine-Translated from a Single Source Language](https://mlsanity.com/paper/2410.23956/) (2024)

## Built on by (most cited first, 12 of 12)

- [Steering LLMs for Culturally Localized Generation](https://mlsanity.com/paper/2603.23301/) (2026)
- [CRaFT: An Explanation-Based Framework for Evaluating Cultural Reasoning in Multilingual Language Models](https://mlsanity.com/paper/2510.14014/) (2025)
- [A Unified Framework to Quantify Cultural Intelligence of AI](https://mlsanity.com/paper/2603.01211/) (2026)
- [The Culture Funnel: You Can't Align What isn't in the Data](https://mlsanity.com/paper/2606.13808/) (2026)
- [Unmasking the Factual-Conceptual Gap in Persian Language Models](https://mlsanity.com/paper/2602.17623/) (2026)
- [The GaoYao Benchmark: A Comprehensive Framework for Evaluating Multilingual and Multicultural Abilities of Large Language Models](https://mlsanity.com/paper/2604.20225/) (2026)
- [MADE: Beyond Scoring via a Multilingual Agentic Diagnosing Engine for Fine-Grained Evaluation Insights](https://mlsanity.com/paper/2606.07020/) (2026)
- [Benchmarking Frontier LLMs on Arabic Cultural and Sociolinguistic Knowledge: A Cross-Evaluation Framework with Human SME Ground Truth](https://mlsanity.com/paper/2607.00139/) (2026)
- [The "Knowledge-Behavior Gap" in Cultural Taboo Safety of Large Language Models](https://mlsanity.com/paper/2608.12341/) (2026)
- [CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systems](https://mlsanity.com/paper/2609.22494/) (2026)
- [Chronologic: Measuring Language Models' Ability to Represent the Past](https://mlsanity.com/paper/2609.23178/) (2026)
- [Large-scale factor analysis shows machine intelligence is only partially interpretable](https://mlsanity.com/paper/2609.36515/) (2026)

## Abstract (arXiv)

As large language models (LLMs) are increasingly deployed in diverse cultural environments, evaluating their cultural understanding capability has become essential for ensuring trustworthy and culturally aligned applications. However, most existing benchmarks lack comprehensiveness and are challenging to scale and adapt across different cultural contexts, because their frameworks often lack guidance from well-established cultural theories and tend to rely on expert-driven manual annotations. To address these issues, we propose CultureScope, the most comprehensive evaluation framework to date for assessing cultural understanding in LLMs. Inspired by the cultural iceberg theory, we design a novel dimensional schema for cultural knowledge classification, comprising 3 layers and 140 dimensions, which guides the automated construction of culture-specific knowledge bases and corresponding evaluation datasets for any given languages and cultures. Experimental results demonstrate that our method can effectively evaluate cultural understanding. They also reveal that existing large language models lack comprehensive cultural competence, and merely incorporating multilingual data does not necessarily enhance cultural understanding. All code and data files are available at https://github.com/HoganZinger/Culture