# Structured Moral Reasoning in Language Models: A Value-Grounded Evaluation Framework

arXiv: https://arxiv.org/abs/2506.14948 · HTML: https://mlsanity.com/paper/2506.14948/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 19 of its 64 references that are in the map.
- First posted on arXiv 17 Jun 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 19 of 19)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [Are NLP Models really able to Solve Simple Math Word Problems?](https://mlsanity.com/paper/2103.07191/) (2021)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [Decomposed Prompting: A Modular Approach for Solving Complex Tasks](https://mlsanity.com/paper/2210.02406/) (2022)
- [2 OLMo 2 Furious](https://mlsanity.com/paper/2501.00656/) (2024)
- [A Survey on Knowledge Distillation of Large Language Models](https://mlsanity.com/paper/2402.13116/) (2024)
- [Social Chemistry 101: Learning to Reason about Social and Moral Norms](https://mlsanity.com/paper/2011.00620/) (2020)
- [Reframing Instructional Prompts to GPTk's Language](https://mlsanity.com/paper/2109.07830/) (2021)
- [Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis](https://mlsanity.com/paper/2304.04675/) (2023)
- [Can Machines Learn Morality? The Delphi Experiment](https://mlsanity.com/paper/2110.07574/) (2021)
- [A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chains](https://mlsanity.com/paper/2402.00559/) (2024)
- [MoralBench: Moral Evaluation of LLMs](https://mlsanity.com/paper/2406.04428/) (2024)
- [Aligning Large Language Models with Human: A Survey](https://mlsanity.com/paper/2307.12966/) (2023)
- [Are Rules Meant to be Broken? Understanding Multilingual Moral Reasoning as a Computational Pipeline with UniMoral](https://mlsanity.com/paper/2502.14083/) (2025)

## Built on by (most cited first, 16 of 16)

- [Large Language Model Reasoning Failures](https://mlsanity.com/paper/2602.06176/) (2026)
- [One Model, Many Morals: Uncovering Cross-Linguistic Misalignments in Computational Moral Reasoning](https://mlsanity.com/paper/2509.21443/) (2025)
- [ProMoral-Bench: Evaluating Prompting Strategies for Moral Reasoning and Safety in LLMs](https://mlsanity.com/paper/2602.13274/) (2026)
- [Pluralistic Alignment for Healthcare: A Role-Driven Framework](https://mlsanity.com/paper/2509.10685/) (2025)
- [Understanding Moral Reasoning Trajectories in Large Language Models: Toward Probing-Based Explainability](https://mlsanity.com/paper/2603.16017/) (2026)
- [Reasoning or Rhetoric? An Empirical Analysis of Moral Reasoning Explanations in Large Language Models](https://mlsanity.com/paper/2603.21854/) (2026)
- [D2VBench: Benchmarking Large Language Models with Value Dilemmas in Daily Scenarios](https://mlsanity.com/paper/2607.19834/) (2026)
- [Survey-to-Behavior: Downstream Alignment of Human Values in LLMs via Survey Questions](https://mlsanity.com/paper/2508.11414/) (2025)
- [Moral Lenses, Political Coordinates: Towards Ideological Positioning of Morally Conditioned LLMs](https://mlsanity.com/paper/2601.08634/) (2026)
- [Event-Centric Human Value Understanding in News-Domain Texts: An Actor-Conditioned, Multi-Granularity Benchmark](https://mlsanity.com/paper/2603.17838/) (2026)
- [Are You the A-hole? A Fair, Multi-Perspective Ethical Reasoning Framework](https://mlsanity.com/paper/2605.00270/) (2026)
- [Where Paths Split: Localized, Calibrated Control of Moral Reasoning in Large Language Models](https://mlsanity.com/paper/2605.03609/) (2026)
- [Are LLMs Bad at Moral Reasoning?](https://mlsanity.com/paper/2606.11635/) (2026)
- [MET: Theory-Grounded and Culture-Aware Multilingual Moral Reasoning](https://mlsanity.com/paper/2607.11736/) (2026)
- [Appearing Legitimate is Not Enough: Interrogating Synthetic Agents in Representational Processes through a Participatory Design Lens](https://mlsanity.com/paper/2608.17099/) (2026)
- [How do LLMs Evaluate Perceived Moral Agency? Investigating Moral Decision-Making in Human-Artificial Agents Interactions](https://mlsanity.com/paper/2609.05037/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly deployed in domains requiring moral understanding, yet their reasoning often remains shallow, and misaligned with human reasoning. Unlike humans, whose moral reasoning integrates contextual trade-offs, value systems, and ethical theories, LLMs often rely on surface patterns, leading to biased decisions in morally and ethically complex scenarios. To address this gap, we present a value-grounded framework for evaluating and distilling structured moral reasoning in LLMs. We benchmark 12 open-source models across four moral datasets using a taxonomy of prompts grounded in value systems, ethical theories, and cognitive reasoning strategies. Our evaluation is guided by four questions: (1) Does reasoning improve LLM decision-making over direct prompting? (2) Which types of value/ethical frameworks most effectively guide LLM reasoning? (3) Which cognitive reasoning strategies lead to better moral performance? (4) Can small-sized LLMs acquire moral competence through distillation? We find that prompting with explicit moral structure consistently improves accuracy and coherence, with first-principles reasoning and Schwartz's + care-ethics scaffolds yielding the strongest gains. Furthermore, our supervised distillation approach transfers moral competence from large to small models without additional inference cost. Together, our results offer a scalable path toward interpretable and value-grounded models.