# Analysing Moral Bias in Finetuned LLMs through Mechanistic Interpretability

arXiv: https://arxiv.org/abs/2510.12229 · HTML: https://mlsanity.com/paper/2510.12229/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Probing deception, bias and beliefs, one of 13 directions in Mechanistic interpretability.
- Builds on 19 of its 45 references that are in the map.
- First posted on arXiv 14 Oct 2025.

## Placed in

- Mechanistic interpretability / [Probing deception, bias and beliefs](https://mlsanity.com/mechinterp/probing-deception-bias-and-beliefs/)

## Builds on (most cited first, 19 of 19)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [BERT Rediscovers the Classical NLP Pipeline](https://mlsanity.com/paper/1905.05950/) (2019)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [Towards Best Practices of Activation Patching in Language Models: Metrics and Methods](https://mlsanity.com/paper/2309.16042/) (2023)
- [Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Do](https://mlsanity.com/paper/2103.11790/) (2021)
- [Analyzing Transformers in Embedding Space](https://mlsanity.com/paper/2209.02535/) (2022)
- [Moral Foundations of Large Language Models](https://mlsanity.com/paper/2310.15337/) (2023)
- [Could a Large Language Model be Conscious?](https://mlsanity.com/paper/2303.07103/) (2023)
- [Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Bias](https://mlsanity.com/paper/2308.00225/) (2023)
- [Dissecting Bias in LLMs: A Mechanistic Interpretability Perspective](https://mlsanity.com/paper/2506.05166/) (2025)
- [Exploiting Primacy Effect To Improve Large Language Models](https://mlsanity.com/paper/2507.13949/) (2025)
- [Improving Language Models for Emotion Analysis: Insights from Cognitive Science](https://mlsanity.com/paper/2406.10265/) (2024)
- [Interpreting Bias in Large Language Models: A Feature-Based Approach](https://mlsanity.com/paper/2406.12347/) (2024)
- ["Pull or Not to Pull?'': Investigating Moral Biases in Leading Large Language Models Across Ethical Dilemmas](https://mlsanity.com/paper/2508.07284/) (2025)
- [The Morality of Probability: How Implicit Moral Biases in LLMs May Shape the Future of Human-AI Symbiosis](https://mlsanity.com/paper/2509.10297/) (2025)

## Built on by (most cited first, 2 of 2)

- [Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models](https://mlsanity.com/paper/2601.14004/) (2026)
- [Mechanistic Interpretability of Cognitive Complexity in LLMs via Linear Probing using Bloom's Taxonomy](https://mlsanity.com/paper/2602.17229/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have been shown to internalize human-like biases during finetuning, yet the mechanisms by which these biases manifest remain unclear. In this work, we investigated whether the well-known Knobe effect, a moral bias in intentionality judgements, emerges in finetuned LLMs and whether it can be traced back to specific components of the model. We conducted a Layer-Patching analysis across 3 open-weights LLMs and demonstrated that the bias is not only learned during finetuning but also localized in a specific set of layers. Surprisingly, we found that patching activations from the corresponding pretrained model into just a few critical layers is sufficient to eliminate the effect. Our findings offer new evidence that social biases in LLMs can be interpreted, localized, and mitigated through targeted interventions, without the need for model retraining.