# LLM-GLOBE: A Benchmark Evaluating the Cultural Values Embedded in LLM Output

arXiv: https://arxiv.org/abs/2411.06032 · HTML: https://mlsanity.com/paper/2411.06032/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 62 references that are in the map.
- First posted on arXiv 9 Nov 2024.

## Builds on (most cited first, 24 of 24)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [Using cognitive psychology to understand GPT-3](https://mlsanity.com/paper/2206.14576/) (2022)
- [Understanding Social Reasoning in Language Models with Language Models](https://mlsanity.com/paper/2306.15448/) (2023)
- [Personality Traits in Large Language Models](https://mlsanity.com/paper/2307.00184/) (2023)
- [Towards Measuring and Modeling "Culture" in LLMs: A Survey](https://mlsanity.com/paper/2403.15412/) (2024)
- [Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models](https://mlsanity.com/paper/2305.14763/) (2023)
- [Probing Pre-Trained Language Models for Cross-Cultural Differences in Values](https://mlsanity.com/paper/2203.13722/) (2022)
- [Cultural Bias and Cultural Alignment of Large Language Models](https://mlsanity.com/paper/2311.14096/) (2023)
- [Investigating Cultural Alignment of Large Language Models](https://mlsanity.com/paper/2402.13231/) (2024)
- [Jury Learning: Integrating Dissenting Voices into Machine Learning Models](https://mlsanity.com/paper/2202.02950/) (2022)
- [Benchmarking Vision Language Models for Cultural Understanding](https://mlsanity.com/paper/2407.10920/) (2024)
- [Can ChatGPT Assess Human Personalities? A General Evaluation Framework](https://mlsanity.com/paper/2303.01248/) (2023)
- [How Well Do LLMs Represent Values Across Cultures? Empirical Analysis of LLM Responses Based on Hofstede Cultural Dimensions](https://mlsanity.com/paper/2406.14805/) (2024)
- [Knowledge of cultural moral norms in large language models](https://mlsanity.com/paper/2306.01857/) (2023)
- [Diminished Diversity-of-Thought in a Standard Large Language Model](https://mlsanity.com/paper/2302.07267/) (2023)
- [Evaluating Cultural Adaptability of a Large Language Model via Simulation of Synthetic Personas](https://mlsanity.com/paper/2408.06929/) (2024)
- [Do Humans Trust Advice More if it Comes from AI? An Analysis of Human-AI Interactions](https://mlsanity.com/paper/2107.07015/) (2021)
- [Mind meets machine: Unravelling GPT-4's cognitive psychology](https://mlsanity.com/paper/2303.11436/) (2023)
- [CDEval: A Benchmark for Measuring the Cultural Dimensions of Large Language Models](https://mlsanity.com/paper/2311.16421/) (2023)
- [Susceptibility to Influence of Large Language Models](https://mlsanity.com/paper/2303.06074/) (2023)
- [Large language models can replicate cross-cultural differences in personality](https://mlsanity.com/paper/2310.10679/) (2023)
- [Who Wrote this? How Smart Replies Impact Language and Agency in the Workplace](https://mlsanity.com/paper/2210.06470/) (2022)
- [CultiVerse: Towards Cross-Cultural Understanding for Paintings with Large Language Model](https://mlsanity.com/paper/2405.00435/) (2024)

## Built on by (most cited first, 27 of 27)

- [DeepSeek-R1 Thoughtology: Let's think about LLM Reasoning](https://mlsanity.com/paper/2504.07128/) (2025)
- [Mind the Value-Action Gap: Do LLMs Act in Alignment with Their Values?](https://mlsanity.com/paper/2501.15463/) (2025)
- [CARE: Multilingual Human Preference Learning for Cultural Awareness](https://mlsanity.com/paper/2504.05154/) (2025)
- [Hire Your Anthropologist! Rethinking Culture Benchmarks Through an Anthropological Lens](https://mlsanity.com/paper/2510.05931/) (2025)
- [DRISHTIKON: A Multimodal Multilingual Benchmark for Testing Language Models' Understanding on Indian Culture](https://mlsanity.com/paper/2509.19274/) (2025)
- [CultureScope: A Dimensional Lens for Probing Cultural Understanding in LLMs](https://mlsanity.com/paper/2509.16188/) (2025)
- [ValuesRAG: Enhancing Cultural Alignment Through Retrieval-Augmented Contextual Learning](https://mlsanity.com/paper/2501.01031/) (2025)
- [Humanizing LLMs: A Survey of Psychological Measurements with Tools, Datasets, and Human-Agent Applications](https://mlsanity.com/paper/2505.00049/) (2025)
- [CAReDiO: Cultural Alignment via Representativeness and Distinctiveness Guided Data Optimization](https://mlsanity.com/paper/2504.08820/) (2025)
- [Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectives](https://mlsanity.com/paper/2506.09656/) (2025)
- [AdAEM: An Adaptively and Automated Extensible Measurement of LLMs' Value Difference](https://mlsanity.com/paper/2505.13531/) (2025)
- [Identity-Aware Large Language Models require Cultural Reasoning](https://mlsanity.com/paper/2510.18510/) (2025)
- [Human Values Matter: Investigating How Misalignment Shapes Collective Behaviors in LLM Agent Communities](https://mlsanity.com/paper/2604.05339/) (2026)
- [CCD-Bench: Probing Cultural Conflict in Large Language Model Decision-Making](https://mlsanity.com/paper/2510.03553/) (2025)
- [Toward Culturally Aligned LLMs through Ontology-Guided Multi-Agent Reasoning](https://mlsanity.com/paper/2601.21700/) (2026)
- [A Unified Framework to Quantify Cultural Intelligence of AI](https://mlsanity.com/paper/2603.01211/) (2026)
- [Implicit Values Embedded in How Humans and LLMs Complete Subjective Everyday Tasks](https://mlsanity.com/paper/2510.03384/) (2025)
- [Value Alignment Tax: Measuring Value Trade-offs in LLM Alignment](https://mlsanity.com/paper/2602.12134/) (2026)
- [Distributional Open-Ended Evaluation of LLM Cultural Value Alignment Based on Value Codebook](https://mlsanity.com/paper/2604.06210/) (2026)
- [When AI Speaks, Whose Values Does It Express? A Cross-Cultural Audit of Individualism-Collectivism Bias in Large Language Models](https://mlsanity.com/paper/2604.22153/) (2026)
- [BiasLab: A Multilingual Dual-Framing Framework for LLM Bias Measurement, Applied to Workplace and HR Contexts](https://mlsanity.com/paper/2601.06861/) (2026)
- [Beyond Instrumental and Substitutive Paradigms: Introducing Machine Culture as an Emergent Phenomenon in Large Language Models](https://mlsanity.com/paper/2601.17096/) (2026)
- [The Role of Dataset Linguistic Structure in the Cultural Awareness of Large Language Models](https://mlsanity.com/paper/2602.01161/) (2026)
- [HiMed: Incentivizing Hindi Reasoning in Medical LLMs](https://mlsanity.com/paper/2605.24635/) (2026)
- [JuICE: A Benchmark for Evaluating LLM-Judge in Identifying Cultural Errors](https://mlsanity.com/paper/2605.26955/) (2026)
- [FineWeb-CLaR: Culture, Language, and Region Annotations for Benchmark-Aligned Corpus Auditing](https://mlsanity.com/paper/2609.25298/) (2026)
- [Deflecting the Value Compass: Interacting with Large Language Models Temporarily Shifts Human Value Priorities Toward Personal Focus](https://mlsanity.com/paper/2609.25586/) (2026)

## Abstract (arXiv)

Immense effort has been dedicated to minimizing the presence of harmful or biased generative content and better aligning AI output to human intention; however, research investigating the cultural values of LLMs is still in very early stages. Cultural values underpin how societies operate, providing profound insights into the norms, priorities, and decision making of their members. In recognition of this need for further research, we draw upon cultural psychology theory and the empirically-validated GLOBE framework to propose the LLM-GLOBE benchmark for evaluating the cultural value systems of LLMs, and we then leverage the benchmark to compare the values of Chinese and US LLMs. Our methodology includes a novel "LLMs-as-a-Jury" pipeline which automates the evaluation of open-ended content to enable large-scale analysis at a conceptual level. Results clarify similarities and differences that exist between Eastern and Western cultural value systems and suggest that open-generation tasks represent a more promising direction for evaluation of cultural values. We interpret the implications of this research for subsequent model development, evaluation, and deployment efforts as they relate to LLMs, AI cultural alignment more broadly, and the influence of AI cultural value systems on human-AI collaboration outcomes.