# A Survey on Moral Foundation Theory and Pre-Trained Language Models: Current Advances and Challenges

arXiv: https://arxiv.org/abs/2409.13521 · HTML: https://mlsanity.com/paper/2409.13521/

## Key facts

- Cited by 3 papers in the map (data through 13 Oct 2026).
- Builds on 53 of its 103 references that are in the map.
- First posted on arXiv 20 Sep 2024.

## Builds on (most cited first, 30 of 53)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [OpenAssistant Conversations -- Democratizing Large Language Model Alignment](https://mlsanity.com/paper/2304.07327/) (2023)
- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [A Survey of Transformers](https://mlsanity.com/paper/2106.04554/) (2021)
- [Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs](https://mlsanity.com/paper/2307.03393/) (2023)
- [Social Chemistry 101: Learning to Reason about Social and Moral Norms](https://mlsanity.com/paper/2011.00620/) (2020)
- [Evaluating Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2310.19736/) (2023)
- [Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Do](https://mlsanity.com/paper/2103.11790/) (2021)
- [Towards Measuring and Modeling "Culture" in LLMs: A Survey](https://mlsanity.com/paper/2403.15412/) (2024)
- [Cultural Bias and Cultural Alignment of Large Language Models](https://mlsanity.com/paper/2311.14096/) (2023)
- [Can Machines Learn Morality? The Delphi Experiment](https://mlsanity.com/paper/2110.07574/) (2021)
- [The Ghost in the Machine has an American accent: value conflict in GPT-3](https://mlsanity.com/paper/2203.07785/) (2022)
- [UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmark](https://mlsanity.com/paper/2103.13009/) (2021)
- [When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment](https://mlsanity.com/paper/2210.01478/) (2022)
- [The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems](https://mlsanity.com/paper/2204.03021/) (2022)
- [Moral Foundations of Large Language Models](https://mlsanity.com/paper/2310.15337/) (2023)
- [Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identity](https://mlsanity.com/paper/2209.12106/) (2022)
- [POLITICS: Pretraining with Same-story Article Comparison for Ideology Prediction and Stance Detection](https://mlsanity.com/paper/2205.00619/) (2022)
- [SqueezeBERT: What can computer vision teach NLP about efficient neural networks?](https://mlsanity.com/paper/2006.11316/) (2020)

## Built on by (most cited first, 3 of 3)

- [The Fragility Of Moral Judgment In Large Language Models](https://mlsanity.com/paper/2603.05651/) (2026)
- [Fairness Metric Design Exploration in Multi-Domain Moral Sentiment Classification using Transformer-Based Models](https://mlsanity.com/paper/2510.11222/) (2025)
- [Can Valence Reflect Morality in Natural Language? A Preliminary Annotation Study](https://mlsanity.com/paper/2607.20461/) (2026)

## Abstract (arXiv)

Moral values have deep roots in early civilizations, codified within norms and laws that regulated societal order and the common good. They play a crucial role in understanding the psychological basis of human behavior and cultural orientation. The Moral Foundation Theory (MFT) is a well-established framework that identifies the core moral foundations underlying the manner in which different cultures shape individual and social lives. Recent advancements in natural language processing, particularly Pre-trained Language Models (PLMs), have enabled the extraction and analysis of moral dimensions from textual data. This survey presents a comprehensive review of MFT-informed PLMs, providing an analysis of moral tendencies in PLMs and their application in the context of the MFT. We also review relevant datasets and lexicons and discuss trends, limitations, and future directions. By providing a structured overview of the intersection between PLMs and MFT, this work bridges moral psychology insights within the realm of PLMs, paving the way for further research and development in creating morally aware AI systems.