# The Pluralistic Moral Gap: Understanding Judgment and Value Differences between Humans and Large Language Models

arXiv: https://arxiv.org/abs/2507.17216 · HTML: https://mlsanity.com/paper/2507.17216/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 47 references that are in the map.
- First posted on arXiv 23 Jul 2025.

## Builds on (most cited first, 25 of 25)

- [Discovering Language Model Behaviors with Model-Written Evaluations](https://mlsanity.com/paper/2212.09251/) (2022)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models](https://mlsanity.com/paper/2404.18796/) (2024)
- [Evaluating the Moral Beliefs Encoded in LLMs](https://mlsanity.com/paper/2307.14324/) (2023)
- [Do LLMs exhibit human-like response biases? A case study in survey design](https://mlsanity.com/paper/2311.04076/) (2023)
- [Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models](https://mlsanity.com/paper/2402.16786/) (2024)
- [Can Machines Learn Morality? The Delphi Experiment](https://mlsanity.com/paper/2110.07574/) (2021)
- [Collective Constitutional AI: Aligning a Language Model with Public Input](https://mlsanity.com/paper/2406.07814/) (2024)
- [Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaboration](https://mlsanity.com/paper/2406.15951/) (2024)
- [Which Economic Tasks are Performed with AI? Evidence from Millions of Claude Conversations](https://mlsanity.com/paper/2503.04761/) (2025)
- [Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs](https://mlsanity.com/paper/2111.13654/) (2021)
- [Clio: Privacy-Preserving Insights into Real-World AI Use](https://mlsanity.com/paper/2412.13678/) (2024)
- [The AI Review Lottery: Widespread AI-Assisted Peer Reviews Boost Paper Scores and Acceptance Rates](https://mlsanity.com/paper/2405.02150/) (2024)
- [Benchmarking Distributional Alignment of Large Language Models](https://mlsanity.com/paper/2411.05403/) (2024)
- [Are Large Language Models Consistent over Value-laden Questions?](https://mlsanity.com/paper/2407.02996/) (2024)
- [Values in the Wild: Discovering and Analyzing Values in Real-World Language Model Interactions](https://mlsanity.com/paper/2504.15236/) (2025)
- [Quantifying social organization and political polarization in online platforms](https://mlsanity.com/paper/2010.00590/) (2020)
- [Normative Evaluation of Large Language Models with Everyday Moral Dilemmas](https://mlsanity.com/paper/2501.18081/) (2025)
- [MoralBERT: A Fine-Tuned Language Model for Capturing Moral Values in Social Discussions](https://mlsanity.com/paper/2403.07678/) (2024)
- [Language Model Council: Democratically Benchmarking Foundation Models on Highly Subjective Tasks](https://mlsanity.com/paper/2406.08598/) (2024)
- [Evaluating Gender Bias of LLMs in Making Morality Judgements](https://mlsanity.com/paper/2410.09992/) (2024)
- [Spillover of Antisocial Behavior from Fringe Platforms: The Unintended Consequences of Community Banning](https://mlsanity.com/paper/2209.09803/) (2022)
- [Explainable Patterns for Distinction and Prediction of Moral Judgement on Reddit](https://mlsanity.com/paper/2201.11155/) (2022)
- [Does Content Moderation Lead Users Away from Fringe Movements? Evidence from a Recovery Community](https://mlsanity.com/paper/2505.03772/) (2025)

## Built on by (most cited first, 7 of 7)

- [Generative Value Conflicts Reveal LLM Priorities](https://mlsanity.com/paper/2509.25369/) (2025)
- [SWAY: A Counterfactual Computational Linguistic Approach to Measuring and Mitigating Sycophancy](https://mlsanity.com/paper/2604.02423/) (2026)
- [One Model, Many Morals: Uncovering Cross-Linguistic Misalignments in Computational Moral Reasoning](https://mlsanity.com/paper/2509.21443/) (2025)
- [Affective Context Amplifies Sycophancy in LLM Responses](https://mlsanity.com/paper/2608.21242/) (2026)
- [Carrot, stick, or both? Price incentives for sustainable food choice in competitive environments](https://mlsanity.com/paper/2512.13174/) (2025)
- [Are LLMs Bad at Moral Reasoning?](https://mlsanity.com/paper/2606.11635/) (2026)
- [Position: Evaluations of AI Moral Reasoning Still Miss Half of the Picture](https://mlsanity.com/paper/2608.14566/) (2026)

## Abstract (arXiv)

People increasingly rely on Large Language Models (LLMs) for moral advice, which may influence humans' decisions. Yet, little is known about how closely LLMs align with human moral judgments. To address this, we introduce the Moral Dilemma Dataset, a benchmark of 1,618 real-world moral dilemmas paired with a distribution of human moral judgments consisting of a binary evaluation and a free-text rationale. We treat this problem as a pluralistic distributional alignment task, comparing the distributions of LLM and human judgments across dilemmas. We find that models reproduce human judgments only under high consensus; alignment deteriorates sharply when human disagreement increases. In parallel, using a 60-value taxonomy built from 3,783 value expressions extracted from rationales, we show that LLMs rely on a narrower set of moral values than humans. These findings reveal a pluralistic moral gap: a mismatch in both the distribution and diversity of values expressed. To close this gap, we introduce Dynamic Moral Profiling (DMP), a Dirichlet-based sampling method that conditions model outputs on human-derived value profiles. DMP improves alignment by 64.3% and enhances value diversity, offering a step toward more pluralistic and human-aligned moral guidance from LLMs.