# Morality is Non-Binary: Building a Pluralist Moral Sentence Embedding Space using Contrastive Learning

arXiv: https://arxiv.org/abs/2401.17228 · HTML: https://mlsanity.com/paper/2401.17228/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 77 references that are in the map.
- First posted on arXiv 30 Jan 2024.

## Builds on (most cited first, 30 of 32)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [Artificial Intelligence, Values and Alignment](https://mlsanity.com/paper/2001.09768/) (2020)
- [Keystroke Dynamics as Part of Lifelogging](https://mlsanity.com/paper/2010.10144/) (2020)
- [Social Chemistry 101: Learning to Reason about Social and Moral Norms](https://mlsanity.com/paper/2011.00620/) (2020)
- [Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Do](https://mlsanity.com/paper/2103.11790/) (2021)
- [Towards Debiasing Sentence Representations](https://mlsanity.com/paper/2007.08100/) (2020)
- [Can Machines Learn Morality? The Delphi Experiment](https://mlsanity.com/paper/2110.07574/) (2021)
- [Toward a Perspectivist Turn in Ground Truthing for Predictive Computing](https://mlsanity.com/paper/2109.04270/) (2021)
- [Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life Anecdotes](https://mlsanity.com/paper/2008.09094/) (2020)
- [ConveRT: Efficient and Accurate Conversational Representations from Transformers](https://mlsanity.com/paper/1911.03688/) (2019)
- [When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment](https://mlsanity.com/paper/2210.01478/) (2022)
- [ClarifyDelphi: Reinforced Clarification Questions with Defeasibility Rewards for Social and Moral Situations](https://mlsanity.com/paper/2212.10409/) (2022)
- [Is Neural Topic Modelling Better than Clustering? An Empirical Study on Clustering with Contextual Embeddings for Topics](https://mlsanity.com/paper/2204.09874/) (2022)
- [Aligning to Social Norms and Values in Interactive Narratives](https://mlsanity.com/paper/2205.01975/) (2022)
- [MoralStrength: Exploiting a Moral Lexicon and Embedding Similarity for Moral Foundations Prediction](https://mlsanity.com/paper/1904.08314/) (2019)
- [Give Me Convenience and Give Her Death: Who Should Decide What Uses of NLP are Appropriate, and on What Basis?](https://mlsanity.com/paper/2005.13213/) (2020)
- [Acquiring Background Knowledge to Improve Moral Value Prediction](https://mlsanity.com/paper/1709.05467/) (2017)
- [The Moral Debater: A Study on the Computational Generation of Morally Framed Arguments](https://mlsanity.com/paper/2203.14563/) (2022)
- [FrameAxis: Characterizing Microframe Bias and Intensity with Word Embedding](https://mlsanity.com/paper/2002.08608/) (2020)
- [Do Differences in Values Influence Disagreements in Online Discussions?](https://mlsanity.com/paper/2310.15757/) (2023)
- [Values, Ethics, Morals? On the Use of Moral Concepts in NLP Research](https://mlsanity.com/paper/2310.13915/) (2023)
- [Reason Against the Machine: Future Directions for Mass Online Deliberation](https://mlsanity.com/paper/2107.12711/) (2021)
- [Learning to Adapt Domain Shifts of Moral Values via Instance Weighting](https://mlsanity.com/paper/2204.07603/) (2022)
- [Pitfalls in the Evaluation of Sentence Embeddings](https://mlsanity.com/paper/1906.01575/) (2019)

## Built on by (most cited first, 7 of 7)

- [Evaluating Moral Beliefs across LLMs through a Pluralistic Framework](https://mlsanity.com/paper/2411.03665/) (2024)
- [Value Preferences Estimation and Disambiguation in Hybrid Participatory Systems](https://mlsanity.com/paper/2402.16751/) (2024)
- [A Game-Theoretic Negotiation Framework for Cross-Cultural Consensus in LLMs](https://mlsanity.com/paper/2506.13245/) (2025)
- [Fairness Metric Design Exploration in Multi-Domain Moral Sentiment Classification using Transformer-Based Models](https://mlsanity.com/paper/2510.11222/) (2025)
- [The Moral Foundations Weibo Corpus](https://mlsanity.com/paper/2411.09612/) (2024)
- [News is More than a Collection of Facts: Moral Frame Preserving News Summarization](https://mlsanity.com/paper/2504.00657/) (2025)
- [Morality in AI. A plea to embed morality in LLM architectures and frameworks](https://mlsanity.com/paper/2511.20689/) (2025)

## Abstract (arXiv)

Recent advances in NLP show that language models retain a discernible level of knowledge in deontological ethics and moral norms. However, existing works often treat morality as binary, ranging from right to wrong. This simplistic view does not capture the nuances of moral judgment. Pluralist moral philosophers argue that human morality can be deconstructed into a finite number of elements, respecting individual differences in moral judgment. In line with this view, we build a pluralist moral sentence embedding space via a state-of-the-art contrastive learning approach. We systematically investigate the embedding space by studying the emergence of relationships among moral elements, both quantitatively and qualitatively. Our results show that a pluralist approach to morality can be captured in an embedding space. However, moral pluralism is challenging to deduce via self-supervision alone and requires a supervised approach with human labels.