mlsanityData to 7 Oct 2026

Mechanistic Interpretability of Emotion Inference in Large Language Models

arXiv 2502.05489 ↗First posted 8 Feb 2025cs.CL cs.AIcited by 24

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) show promising capabilities in predicting human emotions from text. However, the mechanisms through which these models process emotional stimuli remain largely unexplored. Our study addresses this gap by investigating how autoregressive LLMs infer emotions, showing that emotion representations are functionally localized to specific regions in the model. Our evaluation includes diverse model families and sizes and is supported by robustness checks. We then show that the identified representations are psychologically plausible by drawing on cognitive appraisal theory, a well-established psychological framework positing that emotions emerge from evaluations (appraisals) of environmental stimuli. By causally intervening on construed appraisal concepts, we steer the generation and show that the outputs align with theoretical and intuitive expectations. This work highlights a novel way to causally intervene and precisely shape emotional text generation, potentially benefiting safety and alignment in sensitive affective domains.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 46 of its 88 references that are in the map. First posted on arXiv 8 Feb 2025.

Builds on

46 of 88 references.

References · 46

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  3. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  4. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  5. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  6. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  7. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  8. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  9. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  10. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  11. 2019Designing and Interpreting Probes with Control Taskscited by 608
  12. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
34 more
  1. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 452
  2. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 436
  3. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  4. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  5. 2023PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traitscited by 229
  6. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 225
  7. 2016Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Taskscited by 216
  8. 2024How to use and interpret activation patchingcited by 191
  9. 2013Exploiting Similarities among Languages for Machine Translationcited by 187
  10. 2021Implicit Representations of Meaning in Neural Language Modelscited by 179
  11. 2018Dissecting Contextual Word Embeddings: Architecture and Representationcited by 146
  12. 2023Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchillacited by 145
  13. 2023Large Language Models in Law: A Surveycited by 128
  14. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  15. 2023Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMscited by 121
  16. 2023Circuit Component Reuse Across Tasks in Transformer Language Modelscited by 105
  17. 2018Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Informationcited by 97
  18. 2023Cognitive Reframing of Negative Thoughts through Human-Language Model Interactioncited by 77
  19. 2024BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded Datacited by 49
  20. 2022Dimensional Modeling of Emotions in Text with Appraisal Theories: Corpus Creation, Annotation Reliability, and Predictioncited by 45
  21. 2016Representation of linguistic form and function in recurrent neural networkscited by 39
  22. 2023Instruction-tuning Aligns LLMs to the Human Braincited by 39
  23. 2024Monitoring Latent World States in Language Models with Propositional Probescited by 39
  24. 2024The LLM Language Network: A Neuroscientific Approach for Identifying Causally Task-Relevant Unitscited by 34
  25. 2024Limited Ability of LLMs to Simulate Human Psychological Behaviours: a Psychometric Analysiscited by 30
  26. 2024Beyond Concept Bottleneck Models: How to Make Black Boxes Intervenable?cited by 28
  27. 2023Fine-grained Affective Processing Capabilities Emerging from Large Language Modelscited by 24
  28. 2022Do Transformer Models Show Similar Attention Patterns to Task-Specific Human Gaze?cited by 20
  29. 2024GPT-4 Emulates Average-Human Emotional Cognition from a Third-Person Perspectivecited by 16
  30. 2023Uncovering Meanings of Embeddings via Partial Orthogonalitycited by 15
  31. 2024A Reply to Makelov et al. (2023)'s "Interpretability Illusion" Argumentscited by 15
  32. 2023Investigating Large Language Models' Perception of Emotion Using Appraisal Theorycited by 12
  33. 2024Rediscovering the Latent Dimensions of Personality with Large Language Models as Trait Descriptorscited by 6
  34. 2023The Inner Sentiments of a Thoughtcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 20.

24 papers.

Sort:

By area:

  1. 2026This human study did not involve human subjects: Validating LLM simulations as behavioral evidencecited by 26
  2. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 22
  3. 2025Do LLMs "Feel"? Emotion Circuits Discovery and Controlcited by 13
  4. 2025Explaining Large Language Models with gSMILEcited by 4
  5. 2026How Controllable Are Large Language Models? A Unified Evaluation across Behavioral Granularitiescited by 3
  6. 2026Latent Structure of Affective Representations in Large Language Modelscited by 3
  7. 2026From Latent Signals to Reflection Behavior: Tracing Meta-Cognitive Activation Trajectory in R1-Style LLMscited by 2
  8. 2026Do Emotions Influence Moral Judgment in Large Language Models?cited by 2
  9. 2026CAREBench: Evaluating LLMs' Emotion Understanding by Assessing Cognitive Appraisal Reasoningcited by 2
  10. 2026Knowing without Acting: The Disentangled Geometry of Safety Mechanisms in Large Language Modelscited by 1
  11. 2026Whether, Not Which: Mechanistic Interpretability Reveals Dissociable Affect Reception and Emotion Categorization in LLMscited by 1
  12. 2026Read-Best Is Not Steer-Best: A Probing--Steering Layer Dissociation in Omni-Modal Large Language Modelscited by 1
12 more
  1. 2025Categorical Emotions or Appraisals - Which Emotion Model Explains Argument Convincingness Better?cited by 0
  2. 2025Logic of Montagecited by 0
  3. 2026Mechanistic Decoding of Cognitive Constructs in Large Language Modelscited by 0
  4. 2026Are Emotion and Rhetoric Neurons in LLM? Neuron Recognition and Adaptive Masking for Emotion-Rhetoric Prediction Steeringcited by 0
  5. 2026AIPsy-Affect: A Keyword-Free Clinical Stimulus Battery for Mechanistic Interpretability of Emotion in Language Modelscited by 0
  6. 2026Why Are Some Emotions Harder for LLMs? Uncovering the Causal Mechanisms of Emotion Inference via Sparse Autoencoderscited by 0
  7. 2026Negative Before Positive: Asymmetric Valence Processing in Large Language Modelscited by 0
  8. 2026Relational Intervention During Functional Collapse in Large Language Models: A Lexical-Statistical Ablation and a Structure x Register Factorialcited by 0
  9. 2026A Navigable Manifold of Hypothesized Consciousness-Spectrum States in Language Model Representationscited by 0
  10. 2026Understanding the Impact of Linguistic Realization Choices on LLM Stance with Causal Tracingcited by 0
  11. 2026All four leading LLMs talk more than they listen to personality-verified synthetic help-seekerscited by 0
  12. 2026Interpretable Symptom Vectors for Depression in a Large Language Modelcited by 0