# Auditing the Ethical Logic of Generative AI Models

arXiv: https://arxiv.org/abs/2504.17544 · HTML: https://mlsanity.com/paper/2504.17544/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 71 references that are in the map.
- First posted on arXiv 24 Apr 2025.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [GAIA: a benchmark for General AI Assistants](https://mlsanity.com/paper/2311.12983/) (2023)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [From System 1 to System 2: A Survey of Reasoning Large Language Models](https://mlsanity.com/paper/2502.17419/) (2025)
- [Role-Play with Large Language Models](https://mlsanity.com/paper/2305.16367/) (2023)
- [Social Chemistry 101: Learning to Reason about Social and Moral Norms](https://mlsanity.com/paper/2011.00620/) (2020)
- [Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters](https://mlsanity.com/paper/2212.10001/) (2022)
- [Evaluating Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2310.19736/) (2023)
- [Multi-Step Reasoning with Large Language Models, a Survey](https://mlsanity.com/paper/2407.11511/) (2024)
- [Can Machines Learn Morality? The Delphi Experiment](https://mlsanity.com/paper/2110.07574/) (2021)
- [Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequences](https://mlsanity.com/paper/2012.15738/) (2020)
- [BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practices](https://mlsanity.com/paper/2411.12990/) (2024)
- [Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life Anecdotes](https://mlsanity.com/paper/2008.09094/) (2020)
- [The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems](https://mlsanity.com/paper/2204.03021/) (2022)
- [Mapping global dynamics of benchmark creation and saturation in artificial intelligence](https://mlsanity.com/paper/2203.04592/) (2022)
- [Natural Language Reasoning, A Survey](https://mlsanity.com/paper/2303.14725/) (2023)
- [Rethinking Machine Ethics -- Can LLMs Perform Moral Reasoning through the Lens of Moral Theories?](https://mlsanity.com/paper/2308.15399/) (2023)
- [Probing the Moral Development of Large Language Models through Defining Issues Test](https://mlsanity.com/paper/2309.13356/) (2023)
- [Are Large Language Models Moral Hypocrites? A Study Based on Moral Foundations](https://mlsanity.com/paper/2405.11100/) (2024)
- [Analyzing the Ethical Logic of Eight Large Language Models](https://mlsanity.com/paper/2501.08951/) (2025)
- [Palatable Conceptions of Disembodied Being](https://mlsanity.com/paper/2503.16348/) (2025)

## Built on by (most cited first, 5 of 5)

- [Full-Stack Alignment: Co-Aligning AI and Institutions with Thick Models of Value](https://mlsanity.com/paper/2512.03399/) (2025)
- [The Convergent Ethics of AI? Analyzing Moral Foundation Priorities in Large Language Models with a Multi-Framework Approach](https://mlsanity.com/paper/2504.19255/) (2025)
- ["Amazing, They All Lean Left" -- Analyzing the Political Temperaments of Current LLMs](https://mlsanity.com/paper/2507.08027/) (2025)
- [CogniAlign: Survivability-Grounded Multi-Agent Moral Reasoning for Safe and Transparent AI](https://mlsanity.com/paper/2509.13356/) (2025)
- [Frame-Conditioned Moral Computation in LLaMA 3.1-8B-Instruct: A Mechanistic Interpretability Audit of Ethical Reasoning](https://mlsanity.com/paper/2606.15507/) (2026)

## Abstract (arXiv)

As generative AI models become increasingly integrated into high-stakes domains, the need for robust methods to evaluate their ethical reasoning becomes increasingly important. This paper introduces a five-dimensional audit model -- assessing Analytic Quality, Breadth of Ethical Considerations, Depth of Explanation, Consistency, and Decisiveness -- to evaluate the ethical logic of leading large language models (LLMs). Drawing on traditions from applied ethics and higher-order thinking, we present a multi-battery prompt approach, including novel ethical dilemmas, to probe the models' reasoning across diverse contexts. We benchmark seven major LLMs finding that while models generally converge on ethical decisions, they vary in explanatory rigor and moral prioritization. Chain-of-Thought prompting and reasoning-optimized models significantly enhance performance on our audit metrics. This study introduces a scalable methodology for ethical benchmarking of AI systems and highlights the potential for AI to complement human moral reasoning in complex decision-making contexts.