mlsanityData to 7 Oct 2026

SelfIE: Self-Interpretation of Large Language Model Embeddings

arXiv 2403.10949 ↗First posted 16 Mar 2024cs.CL cs.AI cs.LGcited by 63

Not on the map. Why?

Abstract (arXiv)

How do large language models (LLMs) obtain their answers? The ability to explain and control an LLM's reasoning process is key for reliability, transparency, and future model developments. We propose SelfIE (Self-Interpretation of Embeddings), a framework that enables LLMs to interpret their own embeddings in natural language by leveraging their ability to respond to inquiries about a given passage. Capable of interpreting open-world concepts in the hidden embeddings, SelfIE reveals LLM internal reasoning in cases such as making ethical decisions, internalizing prompt injection, and recalling harmful knowledge. SelfIE's text descriptions on hidden embeddings also open up new avenues to control LLM reasoning. We propose Supervised Control, which allows editing open-ended concepts while only requiring gradient computation of individual layer. We extend RLHF to hidden embeddings and propose Reinforcement Control that erases harmful knowledge in LLM without supervision targets.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). Builds on 31 of its 42 references that are in the map. First posted on arXiv 16 Mar 2024.

Builds on

31 of 42 references.

References · 31

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  4. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2016Pointer Sentinel Mixture Modelscited by 2,992
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  8. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  9. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  10. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  11. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  12. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
19 more
  1. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  2. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  3. 2020Concept Bottleneck Modelscited by 1,022
  4. 2020Quantifying Attention Flow in Transformerscited by 759
  5. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 591
  6. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  7. 2014Understanding Deep Image Representations by Inverting Themcited by 399
  8. 2023Studying Large Language Model Generalization with Influence Functionscited by 311
  9. 2018TextWorld: A Learning Environment for Text-based Gamescited by 304
  10. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 225
  11. 2020Modifying Memories in Transformer Modelscited by 222
  12. 2023Linearity of Relation Decoding in Transformer Language Modelscited by 190
  13. 2023Interpreting CLIP's Image Representation via Text-Based Decompositioncited by 189
  14. 2016Synthesizing the preferred inputs for neurons in neural networks via deep generator networkscited by 181
  15. 2021Implicit Representations of Meaning in Neural Language Modelscited by 179
  16. 2016Generating Visual Explanationscited by 156
  17. 2023Inspecting and Editing Knowledge Representations in Language Modelscited by 140
  18. 2022Natural Language Descriptions of Deep Visual Featurescited by 136
  19. 2022Doubly Right Object Recognition: A Why Prompt for Visual Rationalescited by 29

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 25; 2026: 23.

63 papers.

Sort:

By area:

  1. 2024Automatically Interpreting Millions of Features in Large Language Modelscited by 106
  2. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  3. 2024LatentQA: Teaching LLMs to Decode Activations Into Natural Languagecited by 49
  4. 2024Separating Tongue from Thought: Activation Patching Reveals Language-Agnostic Concept Representations in Transformerscited by 42
  5. 2025Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainerscited by 41
  6. 2024Who's asking? User personas and the mechanics of latent misalignmentcited by 40
  7. 2024One-Step is Enough: Sparse Autoencoders for Text-to-Image Diffusion Modelscited by 30
  8. 2024ICT: Image-Object Cross-Level Trusted Intervention for Mitigating Object Hallucination in Large Vision-Language Modelscited by 28
  9. 2025Training Language Models to Explain Their Own Computationscited by 27
  10. 2025Narrow Finetuning Leaves Clearly Readable Traces in Activation Differencescited by 24
  11. 2024Controllable Context Sensitivity and the Knob Behind Itcited by 22
  12. 2025MetaFaith: Faithful Natural Language Uncertainty Expression in LLMscited by 15
38 more
  1. 2025Aligned but Blind: Alignment Increases Implicit Bias by Reducing Awareness of Racecited by 15
  2. 2024Monet: Mixture of Monosemantic Experts for Transformerscited by 12
  3. 2025RomanLens: The Role Of Latent Romanization In Multilinguality In LLMscited by 11
  4. 2025Beyond the Black Box: Interpretability of LLMs in Financecited by 10
  5. 2026Subliminal Steering: Stronger Encoding of Hidden Signalscited by 10
  6. 2025Learning to Interpret Weight Differences in Language Modelscited by 9
  7. 2025Can LLMs Reason Abstractly Over Math Word Problems Without CoT? Disentangling Abstract Formulation From Arithmetic Computationcited by 8
  8. 2025Do Activation Verbalization Methods Convey Privileged Information?cited by 8
  9. 2026Learning a Generative Meta-Model of LLM Activationscited by 8
  10. 2025LARGO: Latent Adversarial Reflection through Gradient Optimization for Jailbreaking LLMscited by 7
  11. 2025Rep2Text: Decoding Full Text from a Single LLM Token Representationcited by 7
  12. 2025Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Modelscited by 6
  13. 2026Efficient and accurate steering of Large Language Models through attention-guided feature learningcited by 6
  14. 2024Meta-Models: An Architecture for Decoding LLM Behaviors Through Interpreted Embeddings and Natural Languagecited by 5
  15. 2025Designing Role Vectors to Improve LLM Inference Behaviourcited by 5
  16. 2025Mind the Generation Process: Fine-Grained Confidence Estimation During LLM Generationcited by 5
  17. 2025Bias Association Discovery Framework for Open-Ended LLM Generationscited by 4
  18. 2024Self-Control of LLM Behaviors by Compressing Suffix Gradient into Prefix Controllercited by 3
  19. 2024Layer-wise Alignment: Examining Safety Alignment Across Image Encoder Layers in Vision Language Modelscited by 3
  20. 2025Enhancing Conversational Agents with Theory of Mind: Aligning Beliefs, Desires, and Intentions for Human-Like Interactioncited by 3
  21. 2024Causal Abstraction in Model Interpretability: A Compact Surveycited by 2
  22. 2025Beyond External Monitors: Enhancing Transparency of Large Language Models for Easier Monitoringcited by 2
  23. 2025InverseScope: Scalable Activation Inversion for Interpreting Large Language Modelscited by 2
  24. 2026Split Personality Training: Revealing Latent Knowledge Through Alternate Personalitiescited by 2
  25. 2026Alignment Reduces Expressed but Not Encoded Gender Bias: A Unified Framework and Studycited by 2
  26. 2026Universal Activation Verbalizer: A Unified Framework for Cross-Model Activation Explanationcited by 2
  27. 2025MIMIC: Multimodal Inversion for Model Interpretation and Conceptualizationcited by 1
  28. 2025On the Representations of Entities in Auto-regressive Large Language Modelscited by 1
  29. 2025DePass: Unified Feature Attributing by Simple Decomposed Forward Passcited by 1
  30. 2026Language Models Can Explain Visual Features via Steeringcited by 1
  31. 2024Unveiling LLM Mechanisms Through Neural ODEs and Control Theorycited by 0
  32. 2024Unraveling Text Generation in LLMs: A Stochastic Differential Equation Approachcited by 0
  33. 2025NeuroFaith: Evaluating LLM Self-Explanation Faithfulness via Internal Representation Alignmentcited by 0
  34. 2025Query Circuits: Explaining How Language Models Answer User Promptscited by 0
  35. 2025TRANSPORTER: Transferring Visual Semantics from VLM Manifoldscited by 0
  36. 2026Learning Self-Interpretation from Interpretability Artifacts: Training Lightweight Adapters on Vector-Label Pairscited by 0
  37. 2026KA2L: A Knowledge-Aware Active Learning Framework for LLMscited by 0
  38. 2026The Master Key Hypothesis: Unlocking Cross-Model Capability Transfer via Linear Subspace Alignmentcited by 0

50 most cited of 63 All 63 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Models
  2. 2024Self-controller: Controlling LLMs with Multi-round Step-by-step Self-awareness
  3. 2026Can We Break LLMs Out of Self-Loops? Fine-Grained Reasoning Control with Activation Steering
  4. 2024Efficient Self-Improvement in Multimodal Large Language Models: A Model-Level Judge-Free Approach
  5. 2023Large Language Models Cannot Self-Correct Reasoning Yet
  6. 2023Large Language Models Suffer From Their Own Output: An Analysis of the Self-Consuming Training Loop
  7. 2025Semantic Voting: A Self-Evaluation-Free Approach for Efficient LLM Self-Improvement on Unverifiable Open-ended Tasks
  8. 2025SelfJudge: Faster Speculative Decoding via Self-Supervised Judge Verification