mlsanityData to 7 Oct 2026

EvalMORAAL: Interpretable Chain-of-Thought and LLM-as-Judge Evaluation for Moral Alignment in Large Language Models

arXiv 2510.05942 ↗First posted 7 Oct 2025cs.CL cs.AIcited by 6

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

We present EvalMORAAL, a transparent chain-of-thought (CoT) framework that uses two scoring methods (log-probabilities and direct ratings) plus a model-as-judge peer review to evaluate moral alignment in 20 large language models. We assess models on the World Values Survey (55 countries, 19 topics) and the PEW Global Attitudes Survey (39 countries, 8 topics). With EvalMORAAL, top models align closely with survey responses (Pearson's r ≈ 0.90 on WVS). Yet we find a clear regional difference: Western regions average r=0.82 while non-Western regions average r=0.61 (a 0.21 absolute gap), indicating a persistent regional alignment gap. Our framework adds three parts: (1) two scoring methods for all models to enable fair comparison, (2) a structured CoT protocol with self-consistency checks, and (3) a model-as-judge peer review that flags 348 conflicts using a data-driven threshold. Peer agreement relates to WVS survey alignment (r=0.74, p<.001; PEW r=0.39, n.s.), supporting automated quality checks. These results show real progress toward culture-aware AI while highlighting open challenges for use across regions.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 15 of its 52 references that are in the map. First posted on arXiv 7 Oct 2025.

Builds on

15 of 52 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025One Model, Many Morals: Uncovering Cross-Linguistic Misalignments in Computational Moral Reasoning25 Sep

Other references · 14

  1. 2022The Ghost in the Machine has an American accent: value conflict in GPT-3cited by 105
  2. 2021A Survey on Gender Bias in Natural Language Processingcited by 89
  3. 2024DailyDilemmas: Revealing Value Preferences of LLMs with Quandaries of Daily Lifecited by 57
  4. 2024How Well Do LLMs Represent Values Across Cultures? Empirical Analysis of LLM Responses Based on Hofstede Cultural Dimensionscited by 51
  5. 2023Assessing LLMs for Moral Value Pluralismcited by 34
  6. 2024Chinese Tiny LLM: Pretraining a Chinese-Centric Large Language Modelcited by 25
  7. 2025Normative Evaluation of Large Language Models with Everyday Moral Dilemmascited by 23
  8. 2025LLM Ethics Benchmark: A Three-Dimensional Assessment System for Evaluating Moral Reasoning in Large Language Modelscited by 20
  9. 2024Beyond Human Norms: Unveiling Unique Values of Large Language Models through Interdisciplinary Approachescited by 13
  10. 2025Explainability in Practice: A Survey of Explainable NLP Across Various Domainscited by 11
  11. 2024Contextual Moral Value Alignment Through Context-Based Aggregationcited by 10
  12. 2025Cultural Bias in Large Language Models: Evaluating AI Agents through Moral Questionnairescited by 4
2 more
  1. 2025Do Large Language Models Understand Morality Across Cultures?cited by 4
  2. 2025Exploring Cultural Variations in Moral Judgments with Large Language Modelscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

  1. 2025Rubric-Conditioned LLM Grading: Alignment, Uncertainty, and Robustnesscited by 4
  2. 2026Untangling Input Language from Reasoning Language: A Diagnostic Framework for Cross-Lingual Moral Alignment in LLMscited by 1
  3. 2026Do LLMs Hold Their Values? MANTA: A Multi-Turn Adversarial Benchmark for Animal Welfare Reasoningcited by 1
  4. 2026trajectory-judge: What Outcome-Only LLM Judges Miss on Agent Trajectoriescited by 1
  5. 2026Crisis-Bench: Benchmarking Strategic Ambiguity and Reputation Management in Large Language Modelscited by 0
  6. 2026How do LLMs Evaluate Perceived Moral Agency? Investigating Moral Decision-Making in Human-Artificial Agents Interactionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Value Benchmarks: Measuring Value-Structure Alignment in Large Language Models via Symmetric Q-Sorts
  2. 2025Differences in the Moral Foundations of Large Language Models
  3. 2026Agreement Is Not Alignment: Divergent Moral Grounds in Human and LLM Ethical Judgments
  4. 2026The Fragility Of Moral Judgment In Large Language Models
  5. 2026Geometry of Values: Task Vector Composition for Ethical Preference Alignment in Language Models
  6. 2026Representational alignment yields generalizable safety in language models
  7. 2024Moral Persuasion in Large Language Models: Evaluating Susceptibility and Ethical Alignment
  8. 2025VAL-Bench: Belief Consistency as a measure for Value Alignment in Language Models