mlsanityData to 7 Oct 2026

The Greatest Good Benchmark: Measuring LLMs' Alignment with Utilitarian Moral Dilemmas

arXiv 2503.19598 ↗First posted 25 Mar 2025cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

The question of how to make decisions that maximise the well-being of all persons is very relevant to design language models that are beneficial to humanity and free from harm. We introduce the Greatest Good Benchmark to evaluate the moral judgments of LLMs using utilitarian dilemmas. Our analysis across 15 diverse LLMs reveals consistently encoded moral preferences that diverge from established moral theories and lay population moral standards. Most LLMs have a marked preference for impartial beneficence and rejection of instrumental harm. These findings showcase the 'artificial moral compass' of LLMs, offering insights into their moral alignment.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 13 of its 24 references that are in the map. First posted on arXiv 25 Mar 2025.

Builds on

13 of 24 references.

References · 13

  1. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  2. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  3. 2020Artificial Intelligence, Values and Alignmentcited by 443
  4. 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 288
  5. 2023From Sparse to Soft Mixtures of Expertscited by 245
  6. 2023Evaluating the Moral Beliefs Encoded in LLMscited by 211
  7. 2022Thinking Fast and Slow in Large Language Modelscited by 136
  8. 2024CogBench: a large language model walks into a psychology labcited by 60
  9. 2023Measuring Inductive Biases of In-Context Learning with Underspecified Demonstrationscited by 49
  10. 2023Values, Ethics, Morals? On the Use of Moral Concepts in NLP Researchcited by 13
  11. 2024Do Moral Judgment and Reasoning Capability of LLMs Change with Language? A Study using the Multilingual Defining Issues Testcited by 9
  12. 2023Towards Theory-based Moral AI: Moral AI with Aggregating Models Based on Normative Ethical Theorycited by 8
1 more
  1. 2023How should the advent of large language models affect the practice of science?cited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 8.

12 papers.

  1. 2025MAEBE: Multi-Agent Emergent Behavior Frameworkcited by 8
  2. 2025Many LLMs Are More Utilitarian Than Onecited by 7
  3. 2025Exploring Cultural Variations in Moral Judgments with Large Language Modelscited by 3
  4. 2026Are Language Models Sensitive to Morally Irrelevant Distractors?cited by 3
  5. 2025Domain Specific Benchmarks for Evaluating Multimodal Large Language Modelscited by 2
  6. 2026Structuring the Space of Sociotechnical Alignmentcited by 2
  7. 2026GAIN: A Benchmark for Goal-Aligned Decision-Making of Large Language Models under Imperfect Normscited by 1
  8. 2026Theory Trace Card: Theory-Driven Socio-Cognitive Evaluation of LLMscited by 0
  9. 2026Probing Ethical Framework Representations in Large Language Models: Structure, Entanglement, and Methodological Challengescited by 0
  10. 2026Staying with the Uncertainty: Uncertainty-Scaffolding Strategies for Artificial Moral Advisors in LLM-to-LLM Simulated Conversationscited by 0
  11. 2026Are LLMs Bad at Moral Reasoning?cited by 0
  12. 2026The yes-no bias of large language models reflects answer order and wording, not shifts in moral judgmentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Beyond Ethical Alignment: Evaluating LLMs as Artificial Moral Assistants
  2. 2025From Stability to Inconsistency: A Study of Moral Preferences in LLMs
  3. 2025Exploring Persona-dependent LLM Alignment for the Moral Machine Experiment
  4. 2024The Moral Turing Test: Evaluating Human-LLM Alignment in Moral Decision-Making
  5. 2023Ethical Reasoning over Moral Alignment: A Case and Framework for In-Context Ethical Policies in LLMs
  6. 2025The Staircase of Ethics: Probing LLM Value Priorities through Multi-Step Induction to Complex Moral Dilemmas
  7. 2025The Morality of Probability: How Implicit Moral Biases in LLMs May Shape the Future of Human-AI Symbiosis
  8. 2023Exploring the psychology of LLMs' Moral and Legal Reasoning