# MoralBERT: A Fine-Tuned Language Model for Capturing Moral Values in Social Discussions

arXiv: https://arxiv.org/abs/2403.07678 · HTML: https://mlsanity.com/paper/2403.07678/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 70 references that are in the map.
- First posted on arXiv 12 Mar 2024.

## Builds on (most cited first, 30 of 31)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [Sustainable AI: Environmental Implications, Challenges and Opportunities](https://mlsanity.com/paper/2111.00364/) (2021)
- [Evaluating the Moral Beliefs Encoded in LLMs](https://mlsanity.com/paper/2307.14324/) (2023)
- [ZeroGen: Efficient Zero-shot Learning via Dataset Generation](https://mlsanity.com/paper/2202.07922/) (2022)
- [Generating Training Data with Language Models: Towards Zero-Shot Language Understanding](https://mlsanity.com/paper/2202.04538/) (2022)
- [Can Machines Learn Morality? The Delphi Experiment](https://mlsanity.com/paper/2110.07574/) (2021)
- [Do We Still Need Clinical Language Models?](https://mlsanity.com/paper/2302.08091/) (2023)
- [LP-MusicCaps: LLM-Based Pseudo Music Captioning](https://mlsanity.com/paper/2307.16372/) (2023)
- [POLITICS: Pretraining with Same-story Article Comparison for Ideology Prediction and Stance Detection](https://mlsanity.com/paper/2205.00619/) (2022)
- [Rethinking Machine Ethics -- Can LLMs Perform Moral Reasoning through the Lens of Moral Theories?](https://mlsanity.com/paper/2308.15399/) (2023)
- [ValueNet: A New Dataset for Human Value Driven Dialogue System](https://mlsanity.com/paper/2112.06346/) (2021)
- [MoralStrength: Exploiting a Moral Lexicon and Embedding Similarity for Moral Foundations Prediction](https://mlsanity.com/paper/1904.08314/) (2019)
- [Moral Framing and Ideological Bias of News](https://mlsanity.com/paper/2009.12979/) (2020)
- [The Moral Foundations Reddit Corpus](https://mlsanity.com/paper/2208.05545/) (2022)
- [Evaluation of ChatGPT Family of Models for Biomedical Reasoning and Classification](https://mlsanity.com/paper/2304.02496/) (2023)
- [Predicting Demographics, Moral Foundations, and Human Values from Digital Behaviors](https://mlsanity.com/paper/1712.01930/) (2017)
- [A Data Fusion Framework for Multi-Domain Morality Learning](https://mlsanity.com/paper/2304.02144/) (2023)
- [Measuring Moral Dimensions in Social Media with Mformer](https://mlsanity.com/paper/2311.10219/) (2023)
- [Learning to Adapt Domain Shifts of Moral Values via Instance Weighting](https://mlsanity.com/paper/2204.07603/) (2022)
- [Chatbot Interaction with Artificial Intelligence: Human Data Augmentation with T5 and Language Transformer Ensemble for Text Classification](https://mlsanity.com/paper/2010.05990/) (2020)
- [Human Values and Attitudes towards Vaccination in Social Media](https://mlsanity.com/paper/1904.00691/) (2019)
- [Moral Narratives Around the Vaccination Debate on Facebook](https://mlsanity.com/paper/2206.01598/) (2022)
- [LibertyMFD: A Lexicon to Assess the Moral Foundation of Liberty](https://mlsanity.com/paper/2209.06750/) (2022)

## Built on by (most cited first, 23 of 23)

- [Many LLMs Are More Utilitarian Than One](https://mlsanity.com/paper/2507.00814/) (2025)
- [The Pluralistic Moral Gap: Understanding Judgment and Value Differences between Humans and Large Language Models](https://mlsanity.com/paper/2507.17216/) (2025)
- [MPCG: Multi-Round Persona-Conditioned Generation for Modeling the Evolution of Misinformation with LLMs](https://mlsanity.com/paper/2509.16564/) (2025)
- [A Survey on Moral Foundation Theory and Pre-Trained Language Models: Current Advances and Challenges](https://mlsanity.com/paper/2409.13521/) (2024)
- [Beyond Human Judgment: A Bayesian Evaluation of LLMs' Moral Values Understanding](https://mlsanity.com/paper/2508.13804/) (2025)
- [Human Values in a Single Sentence: Moral Presence, Hierarchies, and Transformer Ensembles on the Schwartz Continuum](https://mlsanity.com/paper/2601.14172/) (2026)
- [The Moral Gap of Large Language Models](https://mlsanity.com/paper/2507.18523/) (2025)
- [Fairness Metric Design Exploration in Multi-Domain Moral Sentiment Classification using Transformer-Based Models](https://mlsanity.com/paper/2510.11222/) (2025)
- [Identifying and Mitigating Bottlenecks in Role-Playing Agents: A Systematic Study of Disentangling Character Profile Axes](https://mlsanity.com/paper/2601.04716/) (2026)
- [The Consensus Trap: Dissecting Subjectivity and the "Ground Truth" Illusion in Data Annotation](https://mlsanity.com/paper/2602.11318/) (2026)
- [Beyond English: Evaluating Automated Measurement of Moral Foundations in Non-English Discourse with a Chinese Case Study](https://mlsanity.com/paper/2502.02451/) (2025)
- [Implicit Values Embedded in How Humans and LLMs Complete Subjective Everyday Tasks](https://mlsanity.com/paper/2510.03384/) (2025)
- [Moral Semantics Survive Machine Translation: Cross-Lingual Evidence from Moral Foundations Corpora](https://mlsanity.com/paper/2605.22660/) (2026)
- [MoralCLIP: Contrastive Alignment of Vision-and-Language Representations with Moral Foundations Theory](https://mlsanity.com/paper/2506.05696/) (2025)
- [Learning to Diagnose and Correct Moral Errors: Beyond Shallow Heuristics in Moral Alignment](https://mlsanity.com/paper/2601.03079/) (2026)
- [Mapping Election Toxicity on Social Media across Issue, Ideology, and Psychosocial Dimensions](https://mlsanity.com/paper/2604.16765/) (2026)
- [Reinforcing privacy reasoning in LLMs via normative simulacra from fiction](https://mlsanity.com/paper/2604.20904/) (2026)
- [Learning Moral Diversity: Modelling Individual Perspectives in Moral Classification of Texts](https://mlsanity.com/paper/2606.22771/) (2026)
- [Bounded Morality: Defining the Space of Moral Computation](https://mlsanity.com/paper/2607.00002/) (2026)
- [ValueGraph: Value-Signal Guided Graph Pre-training for Contextualized User Representation](https://mlsanity.com/paper/2609.00057/) (2026)
- [Less Is Moral: A CHARMing Framework for Moral Foundations Detection in Endorsement Behaviour](https://mlsanity.com/paper/2609.03330/) (2026)
- [How do LLMs Evaluate Perceived Moral Agency? Investigating Moral Decision-Making in Human-Artificial Agents Interactions](https://mlsanity.com/paper/2609.05037/) (2026)
- [Moral Entropy: Auditing Bias and Uncertainty in Moral Judgment](https://mlsanity.com/paper/2609.21992/) (2026)

## Abstract (arXiv)

Moral values play a fundamental role in how we evaluate information, make decisions, and form judgements around important social issues. Controversial topics, including vaccination, abortion, racism, and sexual orientation, often elicit opinions and attitudes that are not solely based on evidence but rather reflect moral worldviews. Recent advances in Natural Language Processing (NLP) show that moral values can be gauged in human-generated textual content. Building on the Moral Foundations Theory (MFT), this paper introduces MoralBERT, a range of language representation models fine-tuned to capture moral sentiment in social discourse. We describe a framework for both aggregated and domain-adversarial training on multiple heterogeneous MFT human-annotated datasets sourced from Twitter (now X), Reddit, and Facebook that broaden textual content diversity in terms of social media audience interests, content presentation and style, and spreading patterns. We show that the proposed framework achieves an average F1 score that is between 11% and 32% higher than lexicon-based approaches, Word2Vec embeddings, and zero-shot classification with large language models such as GPT-4 for in-domain inference. Domain-adversarial training yields better out-of domain predictions than aggregate training while achieving comparable performance to zero-shot learning. Our approach contributes to annotation-free and effective morality learning, and provides useful insights towards a more comprehensive understanding of moral narratives in controversial social debates using NLP.