mlsanityData to 14 Oct 2026

Auditing the Ethical Logic of Generative AI Models

arXiv 2504.17544 ↗First posted 24 Apr 2025cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

As generative AI models become increasingly integrated into high-stakes domains, the need for robust methods to evaluate their ethical reasoning becomes increasingly important. This paper introduces a five-dimensional audit model -- assessing Analytic Quality, Breadth of Ethical Considerations, Depth of Explanation, Consistency, and Decisiveness -- to evaluate the ethical logic of leading large language models (LLMs). Drawing on traditions from applied ethics and higher-order thinking, we present a multi-battery prompt approach, including novel ethical dilemmas, to probe the models' reasoning across diverse contexts. We benchmark seven major LLMs finding that while models generally converge on ethical decisions, they vary in explanatory rigor and moral prioritization. Chain-of-Thought prompting and reasoning-optimized models significantly enhance performance on our audit metrics. This study introduces a scalable methodology for ethical benchmarking of AI systems and highlights the potential for AI to complement human moral reasoning in complex decision-making contexts.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 28 of its 71 references that are in the map. First posted on arXiv 24 Apr 2025.

Builds on

28 of 71 references.

References · 28

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  6. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  7. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  8. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  9. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  10. 2020Aligning AI With Shared Human Valuescited by 723
  11. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
  12. 2023Role-Play with Large Language Modelscited by 294
16 more
  1. 2020Social Chemistry 101: Learning to Reason about Social and Moral Normscited by 255
  2. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matterscited by 222
  3. 2023Evaluating Large Language Models: A Comprehensive Surveycited by 202
  4. 2024Multi-Step Reasoning with Large Language Models, a Surveycited by 143
  5. 2021Can Machines Learn Morality? The Delphi Experimentcited by 141
  6. 2020Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequencescited by 135
  7. 2024BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practicescited by 114
  8. 2020Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life Anecdotescited by 101
  9. 2022The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systemscited by 93
  10. 2022Mapping global dynamics of benchmark creation and saturation in artificial intelligencecited by 71
  11. 2023Natural Language Reasoning, A Surveycited by 61
  12. 2023Rethinking Machine Ethics -- Can LLMs Perform Moral Reasoning through the Lens of Moral Theories?cited by 36
  13. 2023Probing the Moral Development of Large Language Models through Defining Issues Testcited by 25
  14. 2024Are Large Language Models Moral Hypocrites? A Study Based on Moral Foundationscited by 19
  15. 2025Analyzing the Ethical Logic of Eight Large Language Modelscited by 5
  16. 2025Palatable Conceptions of Disembodied Beingcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 1.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024Informed AI Regulation: Comparing the Ethical Frameworks of Leading LLM Chatbots Using an Ethics-Based Audit to Assess Moral Reasoning and Normative Values
  2. 2024MoralBench: Moral Evaluation of LLMs
  3. 2026Beyond Binary Moral Judgment: Modeling Ethical Pluralism in AI
  4. 2025Beyond Ethical Alignment: Evaluating LLMs as Artificial Moral Assistants
  5. 2024Mapping the Ethics of Generative AI: A Comprehensive Scoping Review
  6. 2025Are Language Models Consequentialist or Deontological Moral Reasoners?
  7. 2024GreedLlama: Performance of Financial Value-Aligned Large Language Models in Moral Reasoning
  8. 2025Diverse Human Value Alignment for Large Language Models via Ethical Reasoning