# BERT has a Moral Compass: Improvements of ethical and moral values of machines

arXiv: https://arxiv.org/abs/1912.05238 · HTML: https://mlsanity.com/paper/1912.05238/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Dec 2019.

## Built on by (most cited first, 21 of 21)

- [Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Do](https://mlsanity.com/paper/2103.11790/) (2021)
- [The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Values](https://mlsanity.com/paper/2310.07629/) (2023)
- [Probing the Moral Development of Large Language Models through Defining Issues Test](https://mlsanity.com/paper/2309.13356/) (2023)
- [The Energy Loss Phenomenon in RLHF: A New Perspective on Mitigating Reward Hacking](https://mlsanity.com/paper/2501.19358/) (2025)
- [Zero-Shot Learners for Natural Language Understanding via a Unified Multiple Choice Perspective](https://mlsanity.com/paper/2210.08590/) (2022)
- [Values, Ethics, Morals? On the Use of Moral Concepts in NLP Research](https://mlsanity.com/paper/2310.13915/) (2023)
- [Are Rules Meant to be Broken? Understanding Multilingual Moral Reasoning as a Computational Pipeline with UniMoral](https://mlsanity.com/paper/2502.14083/) (2025)
- [Decoding Multilingual Moral Preferences: Unveiling LLM's Biases Through the Moral Machine Experiment](https://mlsanity.com/paper/2407.15184/) (2024)
- [Concept-ROT: Poisoning Concepts in Large Language Models with Model Editing](https://mlsanity.com/paper/2412.13341/) (2024)
- [UniEX: An Effective and Efficient Framework for Unified Information Extraction via a Span-extractive Perspective](https://mlsanity.com/paper/2305.10306/) (2023)
- [Why Representation Engineering Works: A Theoretical and Empirical Study in Vision-Language Models](https://mlsanity.com/paper/2503.22720/) (2025)
- [On LLMs' Internal Representation of Code Correctness](https://mlsanity.com/paper/2512.07404/) (2025)
- [An unsupervised framework for tracing textual sources of moral change](https://mlsanity.com/paper/2109.00608/) (2021)
- [Do Large GPT Models Discover Moral Dimensions in Language Representations? A Topological Study Of Sentence Embeddings](https://mlsanity.com/paper/2309.09397/) (2023)
- [Localizing Persona Representations in LLMs](https://mlsanity.com/paper/2505.24539/) (2025)
- [Towards No.1 in CLUE Semantic Matching Challenge: Pre-trained Language Model Erlangshen with Propensity-Corrected Loss](https://mlsanity.com/paper/2208.02959/) (2022)
- [Measuring a Texts Fairness Dimensions Using Machine Learning Based on Social Psychological Factors](https://mlsanity.com/paper/2111.00086/) (2021)
- [Spectral Principal Paths: A Spectral Perspective on Linear Representation Formation in LLMs](https://mlsanity.com/paper/2506.08543/) (2025)
- [MindCraft: How Concept Trees Take Shape In Deep Models](https://mlsanity.com/paper/2510.03265/) (2025)
- [Keep Calm and Avoid Harmful Content: Concept Alignment and Latent Manipulation Towards Safer Answers](https://mlsanity.com/paper/2510.12672/) (2025)
- [Morality in AI. A plea to embed morality in LLM architectures and frameworks](https://mlsanity.com/paper/2511.20689/) (2025)

## Abstract (arXiv)

Allowing machines to choose whether to kill humans would be devastating for world peace and security. But how do we equip machines with the ability to learn ethical or even moral choices? Jentzsch et al.(2019) showed that applying machine learning to human texts can extract deontological ethical reasoning about "right" and "wrong" conduct by calculating a moral bias score on a sentence level using sentence embeddings. The machine learned that it is objectionable to kill living beings, but it is fine to kill time; It is essential to eat, yet one might not eat dirt; it is important to spread information, yet one should not spread misinformation. However, the evaluated moral bias was restricted to simple actions -- one verb -- and a ranking of actions with surrounding context. Recently BERT ---and variants such as RoBERTa and SBERT--- has set a new state-of-the-art performance for a wide range of NLP tasks. But has BERT also a better moral compass? In this paper, we discuss and show that this is indeed the case. Thus, recent improvements of language representations also improve the representation of the underlying ethical and moral values of the machine. We argue that through an advanced semantic representation of text, BERT allows one to get better insights of moral and ethical values implicitly represented in text. This enables the Moral Choice Machine (MCM) to extract more accurate imprints of moral choices and ethical values.