# Towards Cross-lingual Values Judgment: A Consensus-Pluralism Perspective

arXiv: https://arxiv.org/abs/2602.17283 · HTML: https://mlsanity.com/paper/2602.17283/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 37 references that are in the map.
- First posted on arXiv 19 Feb 2026.

## Builds on (most cited first, 29 of 29)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Kimi K2.5: Visual Agentic Intelligence](https://mlsanity.com/paper/2602.02276/) (2026)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation](https://mlsanity.com/paper/2412.03304/) (2024)
- [Survey of Cultural Awareness in Language Models: Text and Beyond](https://mlsanity.com/paper/2411.00860/) (2024)
- [SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoning](https://mlsanity.com/paper/2309.04766/) (2023)
- [MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation](https://mlsanity.com/paper/2503.10497/) (2025)
- [PolyMath: Evaluating Mathematical Reasoning in Multilingual Contexts](https://mlsanity.com/paper/2504.18428/) (2025)
- [Mind the Value-Action Gap: Do LLMs Act in Alignment with Their Values?](https://mlsanity.com/paper/2501.15463/) (2025)
- [MoralBench: Moral Evaluation of LLMs](https://mlsanity.com/paper/2406.04428/) (2024)
- [Language Model Alignment in Multilingual Trolley Problems](https://mlsanity.com/paper/2407.02273/) (2024)
- [IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models](https://mlsanity.com/paper/2406.03368/) (2024)
- [Flames: Benchmarking Value Alignment of LLMs in Chinese](https://mlsanity.com/paper/2311.06899/) (2023)
- [P-MMEval: A Parallel Multilingual Multitask Benchmark for Consistent Evaluation of LLMs](https://mlsanity.com/paper/2411.09116/) (2024)
- [Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Art](https://mlsanity.com/paper/2406.03930/) (2024)
- [Break the Checkbox: Challenging Closed-Style Evaluations of Cultural Alignment in LLMs](https://mlsanity.com/paper/2502.08045/) (2025)
- [ValueCompass: A Framework for Measuring Contextual Value Alignment Between Human and LLMs](https://mlsanity.com/paper/2409.09586/) (2024)
- [Cross-Lingual Auto Evaluation for Assessing Multilingual LLMs](https://mlsanity.com/paper/2410.13394/) (2024)
- [Disentangling Language and Culture for Evaluating Multilingual Large Language Models](https://mlsanity.com/paper/2505.24635/) (2025)
- [CULEMO: Cultural Lenses on Emotion -- Benchmarking LLMs for Cross-Cultural Emotion Understanding](https://mlsanity.com/paper/2503.10688/) (2025)
- [Revisiting LLM Value Probing Strategies: Are They Robust and Expressive?](https://mlsanity.com/paper/2507.13490/) (2025)
- [Do You Know About My Nation? Investigating Multilingual Language Models' Cultural Literacy Through Factual Knowledge](https://mlsanity.com/paper/2511.00657/) (2025)
- [Social Norms in Cinema: A Cross-Cultural Analysis of Shame, Pride and Prejudice](https://mlsanity.com/paper/2402.11333/) (2024)
- [CultureSynth: A Hierarchical Taxonomy-Guided and Retrieval-Augmented Framework for Cultural Question-Answer Synthesis](https://mlsanity.com/paper/2509.10886/) (2025)
- [Cross-Lingual Transfer of Cultural Knowledge: An Asymmetric Phenomenon](https://mlsanity.com/paper/2506.01675/) (2025)
- [DIWALI: Diversity and Inclusivity aWare cuLture specific Items for India: Dataset and Assessment of LLMs for Cultural Text Adaptation in Indian Context](https://mlsanity.com/paper/2509.17399/) (2025)
- [One Model, Many Morals: Uncovering Cross-Linguistic Misalignments in Computational Moral Reasoning](https://mlsanity.com/paper/2509.21443/) (2025)
- [CCD-Bench: Probing Cultural Conflict in Large Language Model Decision-Making](https://mlsanity.com/paper/2510.03553/) (2025)
- [CALM: Culturally Self-Aware Language Models](https://mlsanity.com/paper/2601.03483/) (2026)

## Abstract (arXiv)

As large language models (LLMs) are employed worldwide, existing evaluation paradigms for their multilingual capabilities primarily focus on factual task performance, neglecting the ability to judge content's deep-level values across multiple languages. To bridge this gap, we first reveal two primary challenges in constructing values judgment benchmarks, cultural diversity and disciplinary complexity, and propose a novel two-stage human-AI collaborative annotation framework to alleviate them. This framework identifies the issue scope and nature, establishes specific annotation criteria, and utilizes multiple LLMs for final review. Building upon this framework, we introduce X-Value, the first textitCross-lingual Values Judgment Benchmark designed to evaluate the capability of LLMs in judging deep-level values of content. X-Value comprises 4,750 Question-Answer pairs across 14 languages, covering 7 major global issue categories, and provides 12 granular annotation metadata to facilitate a rigorous evaluation of model performance. Systematic evaluations of X-Value are conducted across 17 LLMs using distinct prompting strategies. Multi-dimensional analysis of accuracy and F1-scores reveals their limitations in cross-lingual values judgment and indicates performance disparities across categories and languages. This work highlights the urgent need to improve the underlying, values-aware content judgment capability of LLMs.footnoteSamples of X-Value are available at https://huggingface.co/datasets/Whitolf/X-Value.