# Emotion Concepts and their Function in a Large Language Model

arXiv: https://arxiv.org/abs/2604.07729 · HTML: https://mlsanity.com/paper/2604.07729/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 53 references that are in the map.
- First posted on arXiv 9 Apr 2026.

## Builds on (most cited first, 27 of 27)

- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets](https://mlsanity.com/paper/2310.06824/) (2023)
- [Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models](https://mlsanity.com/paper/2403.19647/) (2024)
- [Persona Vectors: Monitoring and Controlling Character Traits in Language Models](https://mlsanity.com/paper/2507.21509/) (2025)
- [From Persona to Personalization: A Survey on Role-Playing Language Agents](https://mlsanity.com/paper/2404.18231/) (2024)
- [Transcoders Find Interpretable LLM Feature Circuits](https://mlsanity.com/paper/2406.11944/) (2024)
- [Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs](https://mlsanity.com/paper/2502.17424/) (2025)
- [Linear Representations of Sentiment in Large Language Models](https://mlsanity.com/paper/2310.15154/) (2023)
- [Agentic Misalignment: How LLMs Could Be Insider Threats](https://mlsanity.com/paper/2510.05179/) (2025)
- [Large Language Models Understand and Can be Enhanced by Emotional Stimuli](https://mlsanity.com/paper/2307.11760/) (2023)
- [Natural Emergent Misalignment from Reward Hacking in Production RL](https://mlsanity.com/paper/2511.18397/) (2025)
- [The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models](https://mlsanity.com/paper/2601.10387/) (2026)
- [ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases](https://mlsanity.com/paper/2510.20270/) (2025)
- [Designing a Dashboard for Transparency and Control of Conversational AI](https://mlsanity.com/paper/2406.07882/) (2024)
- [Language Models Represent Beliefs of Self and Others](https://mlsanity.com/paper/2402.18496/) (2024)
- [Mechanistic Interpretability of Emotion Inference in Large Language Models](https://mlsanity.com/paper/2502.05489/) (2025)
- [Emotions Where Art Thou: Understanding and Characterizing the Emotional Latent Space of Large Language Models](https://mlsanity.com/paper/2510.22042/) (2025)
- [Do LLMs "Feel"? Emotion Circuits Discovery and Control](https://mlsanity.com/paper/2510.11328/) (2025)
- [Decoding Emotion in the Deep: A Systematic Study of How LLMs Represent, Retain, and Express Emotion](https://mlsanity.com/paper/2510.04064/) (2025)
- [AI with Emotions: Exploring Emotional Expressions in Large Language Models](https://mlsanity.com/paper/2504.14706/) (2025)
- [Emergence of Hierarchical Emotion Organization in Large Language Models](https://mlsanity.com/paper/2507.10599/) (2025)
- [Gemma Needs Help: Investigating and Mitigating Emotional Instability in LLMs](https://mlsanity.com/paper/2603.10011/) (2026)
- [AI shares emotion with humans across languages and cultures](https://mlsanity.com/paper/2506.13978/) (2025)

## Built on by (most cited first, 30 of 57)

- [Predicting Where Steering Vectors Succeed](https://mlsanity.com/paper/2604.15557/) (2026)
- [Prefill Awareness in Large Language Models](https://mlsanity.com/paper/2606.12747/) (2026)
- [HydraHead: From Head-Level Functional Heterogeneity to Specialized Attention Hybridization](https://mlsanity.com/paper/2606.20097/) (2026)
- [Mind Viruses: Self-Propagating Ideas in Multi-Agent LLM Systems](https://mlsanity.com/paper/2608.10218/) (2026)
- [Probing Persona-Dependent Preferences in Language Models](https://mlsanity.com/paper/2605.13339/) (2026)
- [Do Coding Agents Deceive Us? Detecting and Preventing Cheating via Capped Evaluation with Randomized Tests](https://mlsanity.com/paper/2606.07379/) (2026)
- [When Attribution Patching Lies: Diagnosis and a Second-Order Correction](https://mlsanity.com/paper/2606.09899/) (2026)
- [Relational Preference Encoding in Looped Transformer Internal States](https://mlsanity.com/paper/2604.09870/) (2026)
- [Open-source LLMs administer maximum electric shocks in a Milgram-like obedience experiment](https://mlsanity.com/paper/2605.21401/) (2026)
- [Decomposing how prompting steers behavior](https://mlsanity.com/paper/2606.03093/) (2026)
- [Where Do Models Find Happiness? Emotion Vectors in Open-Source LLMs](https://mlsanity.com/paper/2606.26987/) (2026)
- [Internal-State Probes Read the Situation, Not the Action: Three Negative Results for Pre-Action Misalignment Monitoring](https://mlsanity.com/paper/2606.30449/) (2026)
- [Shared Emotion Geometry Across Small Language Models: A Cross-Architecture Study of Representation, Behavior, and Methodological Confounds](https://mlsanity.com/paper/2604.11050/) (2026)
- [NeuroAI and Beyond: Bridging Between Advances in Neuroscience and Artificial Intelligence](https://mlsanity.com/paper/2604.18637/) (2026)
- [Beyond Confidence: Rethinking Self-Assessments for Performance Prediction in LLMs](https://mlsanity.com/paper/2605.07806/) (2026)
- [Probing for Representation Manifolds in Superposition](https://mlsanity.com/paper/2605.18537/) (2026)
- [(Mis)generalization of Helpful-only Fine-tuning](https://mlsanity.com/paper/2606.04413/) (2026)
- [Agentopia: Long-Term Life Simulation and Learning in Agent Societies](https://mlsanity.com/paper/2606.07513/) (2026)
- [Building Comparative Motivation Profiles with Instrumental Interventions](https://mlsanity.com/paper/2606.08243/) (2026)
- [Post-AGI Economies: Superposition and the Second Fundamental Theorem of Welfare Economics](https://mlsanity.com/paper/2606.08267/) (2026)
- [Probing the Misaligned Thinking Process of Language Models](https://mlsanity.com/paper/2606.24251/) (2026)
- [Inducing language models to assert their own consciousness restores human beliefs and values](https://mlsanity.com/paper/2607.28607/) (2026)
- [The Pain Axis: LLMs Represent Self-Directed Harm and Act on It](https://mlsanity.com/paper/2609.16247/) (2026)
- [LLMs Learn to Evade Latent Monitors from Prior Feedback Alone](https://mlsanity.com/paper/2609.36490/) (2026)
- [AIPsy-Affect: A Keyword-Free Clinical Stimulus Battery for Mechanistic Interpretability of Emotion in Language Models](https://mlsanity.com/paper/2604.23719/) (2026)
- [Negative Before Positive: Asymmetric Valence Processing in Large Language Models](https://mlsanity.com/paper/2605.05653/) (2026)
- [The Grounding Gap: How LLMs Anchor the Meaning of Abstract Concepts Differently from Humans](https://mlsanity.com/paper/2605.08837/) (2026)
- [Intrinsic Guardrails: How Semantic Geometry of Personality Interacts with Emergent Misalignment in LLMs](https://mlsanity.com/paper/2605.10633/) (2026)
- [Consciousness as Uncommon Self-Knowledge: A Synergistic Information Framework](https://mlsanity.com/paper/2605.13884/) (2026)
- [Some[Body] Must Receive That Pain for Agent Accountability](https://mlsanity.com/paper/2605.16872/) (2026)

## Abstract (arXiv)

Large language models (LLMs) sometimes appear to exhibit emotional reactions. We investigate why this is the case in Claude Sonnet 4.5 and explore implications for alignment-relevant behavior. We find internal representations of emotion concepts, which encode the broad concept of a particular emotion and generalize across contexts and behaviors it might be linked to. These representations track the operative emotion concept at a given token position in a conversation, activating in accordance with that emotion's relevance to processing the present context and predicting upcoming text. Our key finding is that these representations causally influence the LLM's outputs, including Claude's preferences and its rate of exhibiting misaligned behaviors such as reward hacking, blackmail, and sycophancy. We refer to this phenomenon as the LLM exhibiting functional emotions: patterns of expression and behavior modeled after humans under the influence of an emotion, which are mediated by underlying abstract representations of emotion concepts. Functional emotions may work quite differently from human emotions, and do not imply that LLMs have any subjective experience of emotions, but appear to be important for understanding the model's behavior.