mlsanityData to 13 Oct 2026

Training Language Models to Explain Their Own Computations

arXiv 2511.08579 ↗First posted 11 Nov 2025cs.CL cs.AI cs.LGcited by 28

Not on the map. Why?

Abstract (arXiv)

Can language models (LMs) learn to faithfully describe their internal computations? Are they better able to describe themselves than other models? We study the extent to which LMs' privileged access to their own internals can be leveraged to produce new techniques for explaining their behavior. Using existing interpretability techniques as a source of ground truth, we fine-tune LMs to generate natural language descriptions of (1) the information encoded by LM features, (2) the causal structure of LMs' internal activations, and (3) the influence of specific input tokens on LM outputs. When trained with only tens of thousands of example explanations, explainer models exhibit non-trivial generalization to new queries. This generalization appears partly attributable to explainer models' privileged access to their own internals: using a model to explain its own computations generally works better than using a *different* model to explain its computations (even if the explainer model is significantly more capable than the target). Our results suggest not only that LMs can learn to reliably explain their internal computations, but that such explanations offer a scalable complement to existing interpretability methods. Code and data at https://github.com/TransluceAI/introspective-interp

Key facts. Cited by 28 papers in the map (data through 13 Oct 2026). Builds on 39 of its 53 references that are in the map. First posted on arXiv 11 Nov 2025.

Builds on

39 of 53 references.

References · 39

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,231
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,876
  3. 2022Locating and Editing Factual Associations in GPTcited by 2,650
  4. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,360
  5. 2017Understanding Black-box Predictions via Influence Functionscited by 1,729
  6. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,391
  7. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,145
  8. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,126
  9. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,101
  10. 2022Toy Models of Superpositioncited by 950
  11. 2023Progress measures for grokking via mechanistic interpretabilitycited by 831
  12. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 718
27 more
  1. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 711
  2. 2025Reasoning Models Don't Always Say What They Thinkcited by 404
  3. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 386
  4. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 345
  5. 2020Understanding the Role of Individual Units in a Deep Neural Networkcited by 255
  6. 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetycited by 236
  7. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 229
  8. 2023Attribution Patching Outperforms Automated Circuit Discoverycited by 200
  9. 2025Open Problems in Mechanistic Interpretabilitycited by 177
  10. 2024Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanismscited by 145
  11. 2024Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoderscited by 137
  12. 2022Natural Language Descriptions of Deep Visual Featurescited by 136
  13. 2024Automatically Interpreting Millions of Features in Large Language Modelscited by 112
  14. 2024Looking Inward: Language Models Can Learn About Themselves by Introspectioncited by 111
  15. 2024Universal Neurons in GPT2 Language Modelscited by 102
  16. 2024SelfIE: Self-Interpretation of Large Language Model Embeddingscited by 65
  17. 2024LatentQA: Teaching LLMs to Decode Activations Into Natural Languagecited by 51
  18. 2024Designing a Dashboard for Transparency and Control of Conversational AIcited by 44
  19. 2025Language Models Fail to Introspect About Their Knowledge of Languagecited by 29
  20. 2021Rationales for Sequential Predictionscited by 26
  21. 2025Self-Interpretability: LLMs Can Describe Complex Internal Processes that Drive Their Decisionscited by 25
  22. 2025Does It Make Sense to Speak of Introspection in Large Language Models?cited by 24
  23. 2024Efficient Automated Circuit Discovery in Transformers using Contextual Decompositioncited by 23
  24. 2025Privileged Self-Access Matters for Introspection in AIcited by 21
  25. 2023The System Model and the User Model: Exploring AI Dashboard Designcited by 12
  26. 2025Do Activation Verbalization Methods Convey Privileged Information?cited by 8
  27. 2024JoPA:Explaining Large Language Model's Generation via Joint Prompt Attributioncited by 6

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 3; 2026: 25.

28 papers.

Sort:

By area:

  1. 2025Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainerscited by 45
  2. 2025Training LLMs for Honesty via Confessionscited by 21
  3. 2026Counterfactual Simulation Training for Chain-of-Thought Faithfulnesscited by 17
  4. 2026A Positive Case for Faithfulness: LLM Self-Explanations Help Predict Model Behaviorcited by 13
  5. 2025Predictive Concept Decoders: Training Scalable End-to-End Interpretability Assistantscited by 10
  6. 2026Can LLMs Introspect? A Reality Checkcited by 10
  7. 2026Learning a Generative Meta-Model of LLM Activationscited by 8
  8. 2026Emergent Introspection in AI is Content-Agnosticcited by 6
  9. 2026Covert Influence Between Language Modelscited by 5
  10. 2026Verbalizing LLMs' assumptions to explain and control sycophancycited by 4
  11. 2026Hallucinations Undermine Trust; Metacognition is a Way Forwardcited by 4
  12. 2026Split Personality Training: Revealing Latent Knowledge Through Alternate Personalitiescited by 3
16 more
  1. 2026Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervisioncited by 3
  2. 2026Language Models Can Explain Visual Features via Steeringcited by 2
  3. 2026Masked by Consensus: Disentangling Privileged Knowledge in LLM Correctnesscited by 2
  4. 2026Would this change your answer? Evaluating Explanations of LLM Behavior In The Wild with Counterfactual Experimentscited by 2
  5. 2026What Is the Minimum Architecture for Prolepsis? Early Irrevocable Commitment Across Tasks in Small Transformerscited by 1
  6. 2026Reasoning Models Know What's Important, and Encode It in Their Activationscited by 1
  7. 2026Building Better Activation Oraclescited by 1
  8. 2026Decoding Hidden Deception in Reasoning LLMs: Activation Explainers for Deception Auditingcited by 1
  9. 2026Position: It's Time to Optimize LLMs for Self-Consistencycited by 1
  10. 2026Evaluating and Improving LLM Self-Modelingcited by 1
  11. 2026Learning Self-Interpretation from Interpretability Artifacts: Training Lightweight Adapters on Vector-Label Pairscited by 0
  12. 2026Justified or Just Convincing? Error Verifiability as a Dimension of LLM Qualitycited by 0
  13. 2026What LLMs explain is not what they believe: Evaluating explanation sufficiency under models' own input beliefscited by 0
  14. 2026SAEVerbalizer: Generating Explanations for Sparse Autoencoder Features via Representation Verbalizationcited by 0
  15. 2026In-Context Neurofeedback: Can LLMs Control Their Internal Representations through Privileged Access?cited by 0
  16. 2026Self-Reports Do Not Identify Self-Models: An Identifiability Test for Counterfactual Reportscited by 0