# Are LLMs Bad at Moral Reasoning?

arXiv: https://arxiv.org/abs/2606.11635 · HTML: https://mlsanity.com/paper/2606.11635/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 27 of its 37 references that are in the map.
- First posted on arXiv 10 Jun 2026.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 27 of 27)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [Prometheus: Inducing Fine-grained Evaluation Capability in Language Models](https://mlsanity.com/paper/2310.08491/) (2023)
- [Humans or LLMs as the Judge? A Study on Judgement Biases](https://mlsanity.com/paper/2402.10669/) (2024)
- [JudgeBench: A Benchmark for Evaluating LLM-based Judges](https://mlsanity.com/paper/2410.12784/) (2024)
- [A Long Way to Go: Investigating Length Correlations in RLHF](https://mlsanity.com/paper/2310.03716/) (2023)
- [Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?](https://mlsanity.com/paper/2405.05904/) (2024)
- [Can Machines Learn Morality? The Delphi Experiment](https://mlsanity.com/paper/2110.07574/) (2021)
- [MoralBench: Moral Evaluation of LLMs](https://mlsanity.com/paper/2406.04428/) (2024)
- [Ethical Reasoning and Moral Value Alignment of LLMs Depend on the Language we Prompt them in](https://mlsanity.com/paper/2404.18460/) (2024)
- [MoReBench: Evaluating Procedural and Pluralistic Moral Reasoning in Language Models, More than Outcomes](https://mlsanity.com/paper/2510.16380/) (2025)
- [LLM Ethics Benchmark: A Three-Dimensional Assessment System for Evaluating Moral Reasoning in Large Language Models](https://mlsanity.com/paper/2505.00853/) (2025)
- [An Empirical Study of LLM-as-a-Judge: How Design Choices Impact Evaluation Reliability](https://mlsanity.com/paper/2506.13639/) (2025)
- [Attributions toward Artificial Agents in a modified Moral Turing Test](https://mlsanity.com/paper/2406.11854/) (2024)
- [Structured Moral Reasoning in Language Models: A Value-Grounded Evaluation Framework](https://mlsanity.com/paper/2506.14948/) (2025)
- [BiasScope: Towards Automated Detection of Bias in LLM-as-a-Judge Evaluation](https://mlsanity.com/paper/2602.09383/) (2026)
- [EgoNormia: Benchmarking Physical Social Norm Understanding](https://mlsanity.com/paper/2502.20490/) (2025)
- [The Greatest Good Benchmark: Measuring LLMs' Alignment with Utilitarian Moral Dilemmas](https://mlsanity.com/paper/2503.19598/) (2025)
- [Are Language Models Consequentialist or Deontological Moral Reasoners?](https://mlsanity.com/paper/2505.21479/) (2025)
- [Discerning What Matters: A Multi-Dimensional Assessment of Moral Competence in LLMs](https://mlsanity.com/paper/2506.13082/) (2025)
- [MORABLES: A Benchmark for Assessing Abstract Moral Reasoning in LLMs with Fables](https://mlsanity.com/paper/2509.12371/) (2025)
- [The Pluralistic Moral Gap: Understanding Judgment and Value Differences between Humans and Large Language Models](https://mlsanity.com/paper/2507.17216/) (2025)
- [Right vs. Right: Can LLMs Make Tough Choices?](https://mlsanity.com/paper/2412.19926/) (2024)
- [Procedural Dilemma Generation for Evaluating Moral Reasoning in Humans and Language Models](https://mlsanity.com/paper/2404.10975/) (2024)
- [Are Language Models Sensitive to Morally Irrelevant Distractors?](https://mlsanity.com/paper/2602.09416/) (2026)
- [MOSAIC: Unveiling the Moral, Social and Individual Dimensions of Large Language Models](https://mlsanity.com/paper/2603.00048/) (2026)

## Abstract (arXiv)

For highly capable AI systems to operate safely in dynamic, open-ended environments, they must be able to identify, understand, and respond to moral reasons for action, and constrain their behaviour accordingly. A growing body of research aims to evaluate this capacity -- moral competence -- in today's most capable AI systems, recently reaching broadly pessimistic conclusions. One of the most ambitious such papers collects gold-standard human-authored rubrics for evaluating moral reasoning in 1,000 cases, and benchmarks frontier AI models against those rubrics, with underwhelming results. In this paper, we argue that the MoReBench dataset can be redeployed to give a much more optimistic picture of LLMs' moral reasoning (an essential part of moral competence). We show that if, instead of scoring LLMs' responses to these cases against these rubrics, we instead give the LLMs the same task given to humans -- to generate scoring rubrics for the moral analysis of particular cases -- the rubrics they generate are both better calibrated to the human rubrics than their open-ended responses, and, where they differ, plausibly reflect nothing more than the vast dimensionality of most moral problems, as well as highlighting some human departures from the "rubric for creating rubrics". Taking these points into consideration, the MoReBench dataset suggests that LLMs are significantly more capable at moral reasoning than was previously believed.