mlsanityData to 14 Oct 2026

MoReBench: Evaluating Procedural and Pluralistic Moral Reasoning in Language Models, More than Outcomes

arXiv 2510.16380 ↗First posted 18 Oct 2025cs.CL cs.AI cs.CYcited by 23

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

As AI systems progress, we rely more on them to make decisions with us and for us. To ensure that such decisions are aligned with human values, it is imperative for us to understand not only what decisions they make but also how they come to those decisions. Reasoning language models, which provide both final responses and (partially transparent) intermediate thinking traces, present a timely opportunity to study AI procedural reasoning. Unlike math and code problems which often have objectively correct answers, moral dilemmas are an excellent testbed for process-focused evaluation because they allow for multiple defensible conclusions. To do so, we present MoReBench: 1,000 moral scenarios, each paired with a set of rubric criteria that experts consider essential to include (or avoid) when reasoning about the scenarios. MoReBench contains over 23 thousand criteria including identifying moral considerations, weighing trade-offs, and giving actionable recommendations to cover cases on AI advising humans moral decisions as well as making moral decisions autonomously. Separately, we curate MoReBench-Theory: 150 examples to test whether AI can reason under five major frameworks in normative ethics. Our results show that scaling laws and existing benchmarks on math, code, and scientific reasoning tasks fail to predict models' abilities to perform moral reasoning. Models also show partiality towards specific moral frameworks (e.g., Benthamite Act Utilitarianism and Kantian Deontology), which might be side effects of popular training paradigms. Together, these benchmarks advance process-focused reasoning evaluation towards safer and more transparent AI.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 30 of its 59 references that are in the map. First posted on arXiv 18 Oct 2025.

Builds on

30 of 59 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Are Language Models Consequentialist or Deontological Moral Reasoners?27 May
  2. 2025The Staircase of Ethics: Probing LLM Value Priorities through Multi-Step Induction to Complex Moral Dilemmas23 May

Other references · 28

  1. 2020Scaling Laws for Neural Language Modelscited by 6,586
  2. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  3. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  4. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  5. 2016Concrete Problems in AI Safetycited by 1,642
  6. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
  7. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 897
  8. 2020Aligning AI With Shared Human Valuescited by 723
  9. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
  10. 2025Humanity's Last Examcited by 491
  11. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 437
  12. 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 429
16 more
  1. 2025Reasoning Models Don't Always Say What They Thinkcited by 415
  2. 2025Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domainscited by 366
  3. 2025PaperBench: Evaluating AI's Ability to Replicate AI Researchcited by 321
  4. 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetycited by 249
  5. 2023Inverse Scaling: When Bigger Isn't Bettercited by 207
  6. 2021Can Machines Learn Morality? The Delphi Experimentcited by 141
  7. 2024Rule Based Rewards for Language Model Safetycited by 108
  8. 2025Stress Testing Deliberative Alignment for Anti-Scheming Trainingcited by 86
  9. 2025Winning Gold at IMO 2025 with a Model-Agnostic Verification-and-Refinement Pipelinecited by 82
  10. 2025Does Thinking More always Help? Mirage of Test-Time Scaling in Reasoning Modelscited by 53
  11. 2025Search-Time Data Contaminationcited by 19
  12. 2025Will AI Tell Lies to Save Sick Children? Litmus-Testing AI Values Prioritization with AIRiskDilemmascited by 18
  13. 2025CLASH: Evaluating Language Models on Judging High-Stakes Dilemmas from Multiple Perspectivescited by 10
  14. 2025Discerning What Matters: A Multi-Dimensional Assessment of Moral Competence in LLMscited by 10
  15. 2025D-REX: A Benchmark for Detecting Deceptive Reasoning in Large Language Modelscited by 7
  16. 2025Beyond Ethical Alignment: Evaluating LLMs as Artificial Moral Assistantscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 23.

23 papers.

Sort:

By area:

  1. 2026HealthBench Professional: Evaluating Large Language Models on Real Clinician Chatscited by 26
  2. 2026Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failurescited by 10
  3. 2026Positive Alignment: Artificial Intelligence for Human Flourishingcited by 5
  4. 2026Reinforcement Learning Towards Broadly and Persistently Beneficial Modelscited by 4
  5. 2026PERSPECTRA: A Scalable and Configurable Pluralist Benchmark of Perspectives from Argumentscited by 3
  6. 2026Omissive Bias in Religious Representation: Benchmarking LLM Answers to Everyday Ethical Decision-makingcited by 2
  7. 2026Incoherent Values? Probing LLM Preferences Through Parametric Variationcited by 2
  8. 2026Does LLM Alignment Really Need Diversity? An Empirical Study of Adapting RLVR Methods for Moral Reasoningcited by 1
  9. 2026Understanding Moral Reasoning Trajectories in Large Language Models: Toward Probing-Based Explainabilitycited by 1
  10. 2026SCRuB: Social Concept Reasoning under Rubric-Based Evaluationcited by 1
  11. 2026QUBRIC: Co-Designing Queries and Rubrics for RL Beyond Verifiable Rewardscited by 1
  12. 2026Every Act Has Its Price: Compressed Moral Composition in Frontier LLMscited by 1
11 more
  1. 2026MDIA: A Multi-Agent Diagnostic Intelligence Pipeline on HealthBench Professionalcited by 0
  2. 2026Generalistic or Specific Embeddings, Which is Better? An Empirical Study on Search for Clinical Coding in Non-English Languagescited by 0
  3. 2026Are LLMs Bad at Moral Reasoning?cited by 0
  4. 2026Normative Robustness as a Frontier for Non-Verifiable Reasoning in LLMscited by 0
  5. 2026Can LLMs Imagine Moral Alternatives Beyond Binary Dilemmas?cited by 0
  6. 2026MET: Theory-Grounded and Culture-Aware Multilingual Moral Reasoningcited by 0
  7. 2026Engineering Trustworthy Agentic AI for Critical Systemscited by 0
  8. 2026When AI Is Your Pastor: A Benchmark for Theological Triage and Pastoral Guidance in Large Language Modelscited by 0
  9. 2026Position: Evaluations of AI Moral Reasoning Still Miss Half of the Picturecited by 0
  10. 2026Representational alignment yields generalizable safety in language modelscited by 0
  11. 2026Steering Vectors for Competing Ethical Frameworks Are Interchangeable: A Competing-Target Test for Activation Steeringcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Procedural Dilemma Generation for Evaluating Moral Reasoning in Humans and Language Models
  2. 2026Beyond Binary Moral Judgment: Modeling Ethical Pluralism in AI
  3. 2025The Morality of Probability: How Implicit Moral Biases in LLMs May Shape the Future of Human-AI Symbiosis
  4. 2025The Convergent Ethics of AI? Analyzing Moral Foundation Priorities in Large Language Models with a Multi-Framework Approach
  5. 2022When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment
  6. 2026MoralityGym: A Benchmark for Evaluating Hierarchical Moral Alignment in Sequential Decision-Making Agents
  7. 2024MoralBench: Moral Evaluation of LLMs
  8. 2025Auditing the Ethical Logic of Generative AI Models