# Beyond Human Judgment: A Bayesian Evaluation of LLMs' Moral Values Understanding

arXiv: https://arxiv.org/abs/2508.13804 · HTML: https://mlsanity.com/paper/2508.13804/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 33 references that are in the map.
- First posted on arXiv 19 Aug 2025.

## Builds on (most cited first, 5 of 5)

- [MoralBERT: A Fine-Tuned Language Model for Capturing Moral Values in Social Discussions](https://mlsanity.com/paper/2403.07678/) (2024)
- [Measuring Moral Dimensions in Social Media with Mformer](https://mlsanity.com/paper/2311.10219/) (2023)
- [Mapping Topics in 100,000 Real-life Moral Dilemmas](https://mlsanity.com/paper/2203.16762/) (2022)
- [What Twitter Data Tell Us about the Future?](https://mlsanity.com/paper/2308.02035/) (2023)
- [The Moral Gap of Large Language Models](https://mlsanity.com/paper/2507.18523/) (2025)

## Built on by (most cited first, 3 of 3)

- [Moral Semantics Survive Machine Translation: Cross-Lingual Evidence from Moral Foundations Corpora](https://mlsanity.com/paper/2605.22660/) (2026)
- [ConflictBench: Evaluating Human-AI Conflict via Interactive and Visually Grounded Environments](https://mlsanity.com/paper/2603.08024/) (2026)
- [Moral Entropy: Auditing Bias and Uncertainty in Moral Judgment](https://mlsanity.com/paper/2609.21992/) (2026)

## Abstract (arXiv)

How do Large Language Models understand moral dimensions compared to humans? This first large-scale Bayesian evaluation of market-leading language models provides the answer. In contrast to prior work using deterministic ground truth (majority or inclusion rules), we model annotator disagreements to capture both aleatoric uncertainty (inherent human disagreement) and epistemic uncertainty (model domain sensitivity). We evaluated the best language models (Claude Sonnet 4, DeepSeek-V3, Llama 4 Maverick) across 250K+ annotations from nearly 700 annotators in 100K+ texts spanning social networks, news and forums. Our GPU-optimized Bayesian framework processed 1M+ model queries, revealing that AI models typically rank among the top 25% of human annotators, performing much better than average balanced accuracy. Importantly, we find that AI produces far fewer false negatives than humans, highlighting their more sensitive moral detection capabilities.