mlsanityData to 14 Oct 2026

Replicating the Geometry of Emotion Representations in a Base Open-Weights Model

arXiv 2609.22208 ↗First posted 1 Sep 2026cs.CL cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Sofroniew et al. (2026) report that emotion concepts in Claude Sonnet 4.5 are represented as vectors whose geometry mirrors human affect psychology. We replicate the representational core of that study on the base pretrained model google/gemma-2-27b, inheriting every disclosed parameter, resolving unspecified steps by disclosed rules, and changing only the subject model. From 205,200 newly generated Claude Sonnet 4.5 stories matching the original corpus design, we extract 171 emotion vectors and recover the core results. The leading principal components form an affective circumplex (PC1 carries 26.7% of variance against the original study's ~27%, PC2 13.4% against ~14%), emotions cluster into similar intuitive families, and the geometry holds across a broad late-middle band. The valence axis aligns with human norms (r = 0.72, against 0.81) and is stable across scales and depth. Arousal aligns at r = 0.67 (against 0.66) but only at the full 171-emotion scale and late depth, so we do not classify it as replicated. Extending the original analysis, a 46-layer sweep locates a sharp seam at L22-26, where the geometry consolidates and the vocabulary readout becomes legible. An embedding-layer baseline finds much of the geometry already present in the static token embeddings, with arousal as the exception. On top-activating held-out text, the geometry predicts token-level co-activation at r = 0.907. At least 52% of vectors peak on structurally non-conceptual tokens, a measured floor for max-activation confounds. Even when a document contains the vector's emotion word, the peak lands on that word only 6.1% of the time. Because the subject is a base model and the stimuli are Claude-generated fiction, the recovered structure is a property of the pretrained representation of Claude-rendered emotion. The original's causal and assistant-facing analyses are out of scope. Code and data are released.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 57 references that are in the map. First posted on arXiv 1 Sep 2026.

Builds on

27 of 57 references.

References · 27

  1. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  2. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  3. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  4. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  5. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  6. 2023Steering Language Models With Activation Engineeringcited by 922
  7. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  8. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 461
  9. 2023Linear Representations of Sentiment in Large Language Modelscited by 163
  10. 2017Learning to Generate Reviews and Discovering Sentimentcited by 148
  11. 2024The Remarkable Robustness of LLMs: Stages of Inference?cited by 146
  12. 2026Emotion Concepts and their Function in a Large Language Modelcited by 57
15 more
  1. 2025Mechanistic Interpretability of Emotion Inference in Large Language Modelscited by 27
  2. 2025NRC VAD Lexicon v2: Norms for Valence, Arousal, and Dominance for over 55k English Termscited by 27
  3. 2025Common Corpus: The Largest Collection of Ethical Data for LLM Pre-Trainingcited by 20
  4. 2025Emotions Where Art Thou: Understanding and Characterizing the Emotional Latent Space of Large Language Modelscited by 18
  5. 2025Do LLMs "Feel"? Emotion Circuits Discovery and Controlcited by 17
  6. 2026Valence-Arousal Subspace in LLMs: Circular Emotion Geometry and Multi-Behavioral Controlcited by 16
  7. 2025Decoding Emotion in the Deep: A Systematic Study of How LLMs Represent, Retain, and Express Emotioncited by 12
  8. 2026Gemma Needs Help: Investigating and Mitigating Emotional Instability in LLMscited by 7
  9. 2026How's it going? Reinforcement learning in language models recruits a functional welfare axiscited by 6
  10. 2026Latent Structure of Affective Representations in Large Language Modelscited by 4
  11. 2026Functional Emotions or Situational Contexts? A Discriminating Test from the Mythos Preview System Cardcited by 3
  12. 2026Where Do Models Find Happiness? Emotion Vectors in Open-Source LLMscited by 2
  13. 2026Shared Emotion Geometry Across Small Language Models: A Cross-Architecture Study of Representation, Behavior, and Methodological Confoundscited by 1
  14. 2026Why Are Some Emotions Harder for LLMs? Uncovering the Causal Mechanisms of Emotion Inference via Sparse Autoencoderscited by 1
  15. 2026The Echo Amplifies the Knowledge: Somatic Marker Analogues in Language Models via Emotion Vector Re-Injectioncited by 1

Built on by

Not cited yet.