# Informed AI Regulation: Comparing the Ethical Frameworks of Leading LLM Chatbots Using an Ethics-Based Audit to Assess Moral Reasoning and Normative Values

arXiv: https://arxiv.org/abs/2402.01651 · HTML: https://mlsanity.com/paper/2402.01651/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 2 of its 2 references that are in the map.
- First posted on arXiv 9 Jan 2024.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 2 of 2)

- [The What-If Tool: Interactive Probing of Machine Learning Models](https://mlsanity.com/paper/1907.04135/) (2019)
- [Personality Traits in Large Language Models](https://mlsanity.com/paper/2307.00184/) (2023)

## Built on by (most cited first, 5 of 5)

- [Large Language Model Safety: A Holistic Survey](https://mlsanity.com/paper/2412.17686/) (2024)
- [Risks and Opportunities of Open-Source Generative AI](https://mlsanity.com/paper/2405.08597/) (2024)
- [Syntactic Framing Fragility: An Audit of Robustness in LLM Ethical Decisions](https://mlsanity.com/paper/2601.09724/) (2025)
- [Ethics Whitepaper: Whitepaper on Ethical Research into Large Language Models](https://mlsanity.com/paper/2410.19812/) (2024)
- [Framing Instability in LLM Ethical Stance: Auditing Negation Sensitivity in Moral Dilemmas](https://mlsanity.com/paper/2601.21433/) (2026)

## Abstract (arXiv)

With the rise of individual and collaborative networks of autonomous agents, AI is deployed in more key reasoning and decision-making roles. For this reason, ethics-based audits play a pivotal role in the rapidly growing fields of AI safety and regulation. This paper undertakes an ethics-based audit to probe the 8 leading commercial and open-source Large Language Models including GPT-4. We assess explicability and trustworthiness by a) establishing how well different models engage in moral reasoning and b) comparing normative values underlying models as ethical frameworks. We employ an experimental, evidence-based approach that challenges the models with ethical dilemmas in order to probe human-AI alignment. The ethical scenarios are designed to require a decision in which the particulars of the situation may or may not necessitate deviating from normative ethical principles. A sophisticated ethical framework was consistently elicited in one model, GPT-4. Nonetheless, troubling findings include underlying normative frameworks with clear bias towards particular cultural norms. Many models also exhibit disturbing authoritarian tendencies. Code is available at https://github.com/jonchun/llm-sota-chatbots-ethics-based-audit.