# Whose Emotions and Moral Sentiments Do Language Models Reflect?

arXiv: https://arxiv.org/abs/2402.11114 · HTML: https://mlsanity.com/paper/2402.11114/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 56 references that are in the map.
- First posted on arXiv 16 Feb 2024.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Discovering Language Model Behaviors with Model-Written Evaluations](https://mlsanity.com/paper/2212.09251/) (2022)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Improving alignment of dialogue agents via targeted human judgements](https://mlsanity.com/paper/2209.14375/) (2022)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection](https://mlsanity.com/paper/2203.09509/) (2022)
- [From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models](https://mlsanity.com/paper/2305.08283/) (2023)
- [The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientation](https://mlsanity.com/paper/2301.01768/) (2023)
- [Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identity](https://mlsanity.com/paper/2209.12106/) (2022)
- [Simulating Social Media Using Large Language Models to Evaluate Alternative News Feed Algorithms](https://mlsanity.com/paper/2310.05984/) (2023)
- [Legal Syllogism Prompting: Teaching Large Language Models for Legal Judgment Prediction](https://mlsanity.com/paper/2307.08321/) (2023)
- [SpanEmo: Casting Multi-label Emotion Classification as Span-prediction](https://mlsanity.com/paper/2101.10038/) (2021)
- [Moral Framing and Ideological Bias of News](https://mlsanity.com/paper/2009.12979/) (2020)
- [Detecting Perceived Emotions in Hurricane Disasters](https://mlsanity.com/paper/2004.14299/) (2020)
- [#RoeOverturned: Twitter Dataset on the Abortion Rights Controversy](https://mlsanity.com/paper/2302.01439/) (2023)
- [A Data Fusion Framework for Multi-Domain Morality Learning](https://mlsanity.com/paper/2304.02144/) (2023)
- [Using Emotion Embeddings to Transfer Knowledge Between Emotions, Languages, and Annotation Formats](https://mlsanity.com/paper/2211.00171/) (2022)
- [CPL-NoViD: Context-Aware Prompt-based Learning for Norm Violation Detection in Online Communities](https://mlsanity.com/paper/2305.09846/) (2023)
- [Reading Between the Tweets: Deciphering Ideological Stances of Interconnected Mixed-Ideology Communities](https://mlsanity.com/paper/2402.01091/) (2024)
- [Polarized Online Discourse on Abortion: Frames and Hostile Expressions among Liberals and Conservatives](https://mlsanity.com/paper/2311.16831/) (2023)

## Built on by (most cited first, 20 of 20)

- [Programming Refusal with Conditional Activation Steering](https://mlsanity.com/paper/2409.05907/) (2024)
- [Self-Alignment: Improving Alignment of Cultural Values in LLMs via In-Context Learning](https://mlsanity.com/paper/2408.16482/) (2024)
- [Conformity and Social Impact on AI Agents](https://mlsanity.com/paper/2601.05384/) (2026)
- [COMMUNITY-CROSS-INSTRUCT: Unsupervised Instruction Generation for Aligning Large Language Models to Online Communities](https://mlsanity.com/paper/2406.12074/) (2024)
- [Don't Blame the Data, Blame the Model: Understanding Noise and Bias When Learning from Subjective Annotations](https://mlsanity.com/paper/2403.04085/) (2024)
- [The Strong Pull of Prior Knowledge in Large Language Models and Its Impact on Emotion Recognition](https://mlsanity.com/paper/2403.17125/) (2024)
- [STEER-BENCH: A Benchmark for Evaluating the Steerability of Large Language Models](https://mlsanity.com/paper/2505.20645/) (2025)
- [Leveraging Machine Learning to Identify Gendered Stereotypes and Body Image Concerns on Diet and Fitness Online Forums](https://mlsanity.com/paper/2407.03551/) (2024)
- [A Survey on Moral Foundation Theory and Pre-Trained Language Models: Current Advances and Challenges](https://mlsanity.com/paper/2409.13521/) (2024)
- [Culturally Grounded Personas in Large Language Models: Characterization and Alignment with Socio-Psychological Value Frameworks](https://mlsanity.com/paper/2601.22396/) (2026)
- [Improving and Assessing the Fidelity of Large Language Models Alignment to Online Communities](https://mlsanity.com/paper/2408.09366/) (2024)
- ["Amazing, They All Lean Left" -- Analyzing the Political Temperaments of Current LLMs](https://mlsanity.com/paper/2507.08027/) (2025)
- [Word Overuse and Alignment in Large Language Models: The Influence of Learning from Human Feedback](https://mlsanity.com/paper/2508.01930/) (2025)
- [Do Emotions Influence Moral Judgment in Large Language Models?](https://mlsanity.com/paper/2604.19125/) (2026)
- [AI-Associated Lexical Shifts Across 34 Languages: Cross-Lingual Convergence and Diachronic Uptake in News Writing](https://mlsanity.com/paper/2605.25358/) (2026)
- [Structuring the Space of Sociotechnical Alignment](https://mlsanity.com/paper/2607.01250/) (2026)
- [Secret Keepers: The Impact of LLMs on Linguistic Markers of Personal Traits](https://mlsanity.com/paper/2404.00267/) (2024)
- [DVMap: Fine-Grained Pluralistic Value Alignment via High-Consensus Demographic-Value Mapping](https://mlsanity.com/paper/2605.14420/) (2026)
- [Modeling Community Attitude through Reaction Tone: A Human-AI Collaborative Framework for Evaluating LLM Alignment with Linguistic Behaviors in Online Communities](https://mlsanity.com/paper/2605.27388/) (2026)
- [Belief Cascades Drive Persuasion in LLM Agent Networks](https://mlsanity.com/paper/2608.25152/) (2026)

## Abstract (arXiv)

Language models (LMs) are known to represent the perspectives of some social groups better than others, which may impact their performance, especially on subjective tasks such as content moderation and hate speech detection. To explore how LMs represent different perspectives, existing research focused on positional alignment, i.e., how closely the models mimic the opinions and stances of different groups, e.g., liberals or conservatives. However, human communication also encompasses emotional and moral dimensions. We define the problem of affective alignment, which measures how LMs' emotional and moral tone represents those of different groups. By comparing the affect of responses generated by 36 LMs to the affect of Twitter messages, we observe significant misalignment of LMs with both ideological groups. This misalignment is larger than the partisan divide in the U.S. Even after steering the LMs towards specific ideological perspectives, the misalignment and liberal tendencies of the model persist, suggesting a systemic bias within LMs.