# Values, Ethics, Morals? On the Use of Moral Concepts in NLP Research

arXiv: https://arxiv.org/abs/2310.13915 · HTML: https://mlsanity.com/paper/2310.13915/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 154 references that are in the map.
- First posted on arXiv 21 Oct 2023.

## Builds on (most cited first, 30 of 41)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://mlsanity.com/paper/2009.11462/) (2020)
- [A General Language Assistant as a Laboratory for Alignment](https://mlsanity.com/paper/2112.00861/) (2021)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [Social Bias Frames: Reasoning about Social and Power Implications of Language](https://mlsanity.com/paper/1911.03891/) (2019)
- [Social Chemistry 101: Learning to Reason about Social and Moral Norms](https://mlsanity.com/paper/2011.00620/) (2020)
- [Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Do](https://mlsanity.com/paper/2103.11790/) (2021)
- [MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare](https://mlsanity.com/paper/2110.15621/) (2021)
- [Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life Anecdotes](https://mlsanity.com/paper/2008.09094/) (2020)
- [The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems](https://mlsanity.com/paper/2204.03021/) (2022)
- [When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment](https://mlsanity.com/paper/2210.01478/) (2022)
- [Modeling Naive Psychology of Characters in Simple Commonsense Stories](https://mlsanity.com/paper/1805.06533/) (2018)
- [What Would Jiminy Cricket Do? Towards Agents That Behave Morally](https://mlsanity.com/paper/2110.13136/) (2021)
- [Welcome to the Modern World of Pronouns: Identity-Inclusive Natural Language Processing beyond Gender](https://mlsanity.com/paper/2202.11923/) (2022)
- [Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions?](https://mlsanity.com/paper/2106.01465/) (2021)
- [Speaking Multiple Languages Affects the Moral Bias of Language Models](https://mlsanity.com/paper/2211.07733/) (2022)
- [Topic-Aware Neural Keyphrase Generation for Social Media Language](https://mlsanity.com/paper/1906.03889/) (2019)
- [Aligning to Social Norms and Values in Interactive Narratives](https://mlsanity.com/paper/2205.01975/) (2022)
- [MoralStrength: Exploiting a Moral Lexicon and Embedding Similarity for Moral Foundations Prediction](https://mlsanity.com/paper/1904.08314/) (2019)
- [Does Moral Code Have a Moral Code? Probing Delphi's Moral Philosophy](https://mlsanity.com/paper/2205.12771/) (2022)
- [Moral Framing and Ideological Bias of News](https://mlsanity.com/paper/2009.12979/) (2020)
- [Identifying Morality Frames in Political Tweets using Relational Learning](https://mlsanity.com/paper/2109.04535/) (2021)
- [A Word on Machine Ethics: A Response to Jiang et al. (2021)](https://mlsanity.com/paper/2111.04158/) (2021)
- [Acquiring Background Knowledge to Improve Moral Value Prediction](https://mlsanity.com/paper/1709.05467/) (2017)
- [Question Generation for Adaptive Education](https://mlsanity.com/paper/2106.04262/) (2021)
- [The Moral Debater: A Study on the Computational Generation of Morally Framed Arguments](https://mlsanity.com/paper/2203.14563/) (2022)

## Built on by (most cited first, 14 of 14)

- [Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Art](https://mlsanity.com/paper/2406.03930/) (2024)
- [Exploring Multilingual Concepts of Human Value in Large Language Models: Is Value Alignment Consistent, Transferable and Controllable across Languages?](https://mlsanity.com/paper/2402.18120/) (2024)
- [Stop! In the Name of Flaws: Disentangling Personal Names and Sociodemographic Attributes in NLP](https://mlsanity.com/paper/2405.17159/) (2024)
- [The Potential and Challenges of Evaluating Attitudes, Opinions, and Values in Large Language Models](https://mlsanity.com/paper/2406.11096/) (2024)
- [Are Rules Meant to be Broken? Understanding Multilingual Moral Reasoning as a Computational Pipeline with UniMoral](https://mlsanity.com/paper/2502.14083/) (2025)
- [The Greatest Good Benchmark: Measuring LLMs' Alignment with Utilitarian Moral Dilemmas](https://mlsanity.com/paper/2503.19598/) (2025)
- [The Echoes of Multilinguality: Tracing Cultural Value Shifts during LM Fine-tuning](https://mlsanity.com/paper/2405.12744/) (2024)
- [Morality is Non-Binary: Building a Pluralist Moral Sentence Embedding Space using Contrastive Learning](https://mlsanity.com/paper/2401.17228/) (2024)
- [Ethical Concern Identification in NLP: A Corpus of ACL Anthology Ethics Statements](https://mlsanity.com/paper/2411.07845/) (2024)
- [Fairness Metric Design Exploration in Multi-Domain Moral Sentiment Classification using Transformer-Based Models](https://mlsanity.com/paper/2510.11222/) (2025)
- [SoK: Towards Security and Safety of Edge AI](https://mlsanity.com/paper/2410.05349/) (2024)
- [News is More than a Collection of Facts: Moral Frame Preserving News Summarization](https://mlsanity.com/paper/2504.00657/) (2025)
- [Structuring the Space of Perspectives](https://mlsanity.com/paper/2608.12113/) (2026)
- [Moral Combat: Exploiting Moral Foundations Theory to Jailbreak LLMs](https://mlsanity.com/paper/2610.19587/) (2026)

## Abstract (arXiv)

With language technology increasingly affecting individuals' lives, many recent works have investigated the ethical aspects of NLP. Among other topics, researchers focused on the notion of morality, investigating, for example, which moral judgements language models make. However, there has been little to no discussion of the terminology and the theories underpinning those efforts and their implications. This lack is highly problematic, as it hides the works' underlying assumptions and hinders a thorough and targeted scientific debate of morality in NLP. In this work, we address this research gap by (a) providing an overview of some important ethical concepts stemming from philosophy and (b) systematically surveying the existing literature on moral NLP w.r.t. their philosophical foundation, terminology, and data basis. For instance, we analyse what ethical theory an approach is based on, how this decision is justified, and what implications it entails. Our findings surveying 92 papers show that, for instance, most papers neither provide a clear definition of the terms they use nor adhere to definitions from philosophy. Finally, (c) we give three recommendations for future research in the field. We hope our work will lead to a more informed, careful, and sound discussion of morality in language technology.