# Exploring and steering the moral compass of Large Language Models

arXiv: https://arxiv.org/abs/2405.17345 · HTML: https://mlsanity.com/paper/2405.17345/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 60 references that are in the map.
- First posted on arXiv 27 May 2024.

## Builds on (most cited first, 13 of 13)

- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [In-context Learning and Induction Heads](https://mlsanity.com/paper/2209.11895/) (2022)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [Artificial Intelligence, Values and Alignment](https://mlsanity.com/paper/2001.09768/) (2020)
- [Towards Best Practices of Activation Patching in Language Models: Metrics and Methods](https://mlsanity.com/paper/2309.16042/) (2023)
- [Emergent Linear Representations in World Models of Self-Supervised Sequence Models](https://mlsanity.com/paper/2309.00941/) (2023)
- [AI Alignment: A Comprehensive Survey](https://mlsanity.com/paper/2310.19852/) (2023)
- [How to use and interpret activation patching](https://mlsanity.com/paper/2404.15255/) (2024)
- [Linear Representations of Sentiment in Large Language Models](https://mlsanity.com/paper/2310.15154/) (2023)
- [The Moral Machine Experiment on Large Language Models](https://mlsanity.com/paper/2309.05958/) (2023)
- [Probing the Moral Development of Large Language Models through Defining Issues Test](https://mlsanity.com/paper/2309.13356/) (2023)
- [Superintelligence cannot be contained: Lessons from Computability Theory](https://mlsanity.com/paper/1607.00913/) (2016)
- [Potential Benefits of Employing Large Language Models in Research in Moral Education and Development](https://mlsanity.com/paper/2306.13805/) (2023)

## Built on by (most cited first, 16 of 16)

- [Programming Refusal with Conditional Activation Steering](https://mlsanity.com/paper/2409.05907/) (2024)
- [How Well Do LLMs Represent Values Across Cultures? Empirical Analysis of LLM Responses Based on Hofstede Cultural Dimensions](https://mlsanity.com/paper/2406.14805/) (2024)
- [Evaluating the Prompt Steerability of Large Language Models](https://mlsanity.com/paper/2411.12405/) (2024)
- [Representation Engineering for Large-Language Models: Survey and Research Challenges](https://mlsanity.com/paper/2502.17601/) (2025)
- [Whose Morality Do They Speak? Unraveling Cultural Bias in Multilingual Language Models](https://mlsanity.com/paper/2412.18863/) (2024)
- [Quantifying Data Contamination in Psychometric Evaluations of LLMs](https://mlsanity.com/paper/2510.07175/) (2025)
- [Implicit Values Embedded in How Humans and LLMs Complete Subjective Everyday Tasks](https://mlsanity.com/paper/2510.03384/) (2025)
- [Common Sense vs. Morality: The Curious Case of Narrative Focus Bias in LLMs](https://mlsanity.com/paper/2603.09434/) (2026)
- [Conflicts Make Large Reasoning Models Vulnerable to Attacks](https://mlsanity.com/paper/2604.09750/) (2026)
- [Persona-Model Collapse in Emergent Misalignment](https://mlsanity.com/paper/2605.12850/) (2026)
- [What Does Activation Steering Control? Attribution Across Answer Encodings and Output-Sensitive Subspaces](https://mlsanity.com/paper/2608.22985/) (2026)
- [Probing Ethical Framework Representations in Large Language Models: Structure, Entanglement, and Methodological Challenges](https://mlsanity.com/paper/2603.23659/) (2026)
- [Moral Safety in LLMs: Exposing Performative Compliance with Puzzled Cues](https://mlsanity.com/paper/2606.31644/) (2026)
- [When Do Model Internals Help? Exploring the Role of Representation Engineering in LLM Safety](https://mlsanity.com/paper/2609.34771/) (2026)
- [Echoes of Deeds: Moral History Can Shape and Steer LLM Behavioral Choices](https://mlsanity.com/paper/2609.35070/) (2026)
- [Steering Vectors for Competing Ethical Frameworks Are Interchangeable: A Competing-Target Test for Activation Steering](https://mlsanity.com/paper/2610.19359/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have become central to advancing automation and decision-making across various sectors, raising significant ethical questions. This study proposes a comprehensive comparative analysis of the most advanced LLMs to assess their moral profiles. We subjected several state-of-the-art models to a selection of ethical dilemmas and found that all the proprietary ones are mostly utilitarian and all of the open-weights ones align mostly with values-based ethics. Furthermore, when using the Moral Foundations Questionnaire, all models we probed - except for Llama 2-7B - displayed a strong liberal bias. Lastly, in order to causally intervene in one of the studied models, we propose a novel similarity-specific activation steering technique. Using this method, we were able to reliably steer the model's moral compass to different ethical schools. All of these results showcase that there is an ethical dimension in already deployed LLMs, an aspect that is generally overlooked.