Compositional Literary Primitives in Instruction-Tuned LLMs: Cross-Architectural SAE Features for Self, Style, and Affect
On the map: Mechanistic interpretability → Activation steering methods
Abstract (arXiv)
We characterize a compositional architecture of literary primitives in two instruction-tuned large language models (Llama 3.1 8B-Instruct and Gemma 2 9B-IT) via sparse autoencoders on mid-depth residual streams. Four feature classes emerge: naming-gates that promote lexical tokens of a target affect, an eleven-self cluster of first-person register features, stylistic register modulators (show-don't-tell and defamiliarization), and compositional emotions that arise only from multi-feature steering. Under a forced-choice 5-LLM judge panel applied to a 27-category emotion taxonomy (Cowen-Keltner), Llama reaches full 27/27 coverage by combining naming-gates, multi-feature recipes, and single self-feature steering; Gemma reaches 23/27 with adoration as the single residual strict-fail. Under random judging, the per-cell pass probability is on the order of 10-3 and the expected number of two-seed false-positive cells across the catalog is negligible, so the observed coverage is not consistent with chance. A cross-architectural asymmetry sits in the strict-versus-soft judge contrast: on the same generations, judges agree more often on Llama outputs than on Gemma outputs because Llama outputs name the target affect more directly while Gemma outputs evoke it through scene and imagery. Both architectures contain self-features that serve simultaneously as register markers and as emotion emitters, including a single most-RLHF-loaded self-feature per architecture that intensifies the institutional Helper-AI persona at one operating regime and produces affect-categorizable output at the same calibrated coefficient. Methodologically, the paper presents a three-stage validation pipeline (logit-lens, LLM-rate, 5-LLM judge) with documented anti-patterns; the total compute is single-GPU and about 15 minutes per emotion-feature discovery cycle.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability. Builds on 20 of its 30 references that are in the map. First posted on arXiv 11 May 2026.
Builds on
20 of 30 references.
Closest ancestors · 1
Same direction, newest first
Other references · 19
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
- 2024A Survey on LLM-as-a-Judgecited by 1,423
- 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
- 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
- 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
- 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 437
- 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
- 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
- 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 345
7 more
- 2023Role-Play with Large Language Modelscited by 294
- 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 281
- 2023Linear Representations of Sentiment in Large Language Modelscited by 163
- 2024Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoderscited by 143
- 2024Universal Neurons in GPT2 Language Modelscited by 102
- 2025Large Language Models Share Representations of Latent Grammatical Concepts Across Typologically Diverse Languagescited by 59
- 2024Monotonic Representation of Numeric Properties in Language Modelscited by 15
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Some Emotions Run Deeper: Layer-wise Probing and Causal Intervention in Large Language Models
- 2025ELSA: A Style Aligned Dataset for Emotionally Intelligent Language Generation
- 2026The Effectiveness of Style Vectors for Steering Large Language Models: A Human Evaluation
- 2026Extracting and Steering Emotion Representations in Small Language Models: A Methodological Comparison
- 2025MUSE: Manipulating Unified Framework for Synthesizing Emotions in Images via Test-Time Optimization
- 2025Mechanistic Interpretability of Emotion Inference in Large Language Models
- 2026From Context Shift to Stylistic Collapse: Why Training Objectives Matter More Than Scale
- 2025AI shares emotion with humans across languages and cultures