mlsanityData to 14 Oct 2026

Eliciting Latent Predictions from Transformers with the Tuned Lens

arXiv 2303.08112 ↗First posted 14 Mar 2023cs.LGcited by 623

On the map: Mechanistic interpretability → Knowledge and representations inside LLMs

Abstract (arXiv)

We analyze transformers from the perspective of iterative inference, seeking to understand how model predictions are refined layer by layer. To do so, we train an affine probe for each block in a frozen pretrained model, making it possible to decode every hidden state into a distribution over the vocabulary. Our method, the tuned lens, is a refinement of the earlier "logit lens" technique, which yielded useful insights but is often brittle. We test our method on various autoregressive language models with up to 20B parameters, showing it to be more predictive, reliable and unbiased than the logit lens. With causal experiments, we show the tuned lens uses similar features to the model itself. We also find the trajectory of latent predictions can be used to detect malicious inputs with high accuracy. All code needed to reproduce our results can be found at https://github.com/AlignmentResearch/tuned-lens.

Key facts. Cited by 623 papers in the map (data through 14 Oct 2026). Placed in Knowledge and representations inside LLMs, one of 13 directions in Mechanistic interpretability. Builds on 42 of its 77 references that are in the map. First posted on arXiv 14 Mar 2023.

Builds on

42 of 77 references.

References · 42

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  6. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  7. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  8. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  9. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  10. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
  11. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  12. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
30 more
  1. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  2. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  3. 2016Deep Networks with Stochastic Depthcited by 979
  4. 2022Toy Models of Superpositioncited by 971
  5. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  6. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
  7. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 713
  8. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 678
  9. 2019Designing and Interpreting Probes with Control Taskscited by 640
  10. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  11. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 532
  12. 2019Reducing Transformer Depth on Demand with Structured Dropoutcited by 439
  13. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 431
  14. 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 332
  15. 2022Confident Adaptive Language Modelingcited by 310
  16. 2014Understanding image representations by measuring their equivariance and equivalencecited by 243
  17. 2021Revisiting Model Stitching to Compare Neural Representationscited by 209
  18. 2021All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Qualitycited by 156
  19. 2021Deep Learning Through the Lens of Example Difficultycited by 141
  20. 2022Analyzing Transformers in Embedding Spacecited by 132
  21. 2021BERT Busters: Outlier Dimensions that Disrupt Transformerscited by 112
  22. 2020Accelerating Training of Transformer-Based Language Models with Progressive Layer Droppingcited by 93
  23. 2021Similarity and Matching of Neural Network Representationscited by 78
  24. 2017Residual Connections Encourage Iterative Inferencecited by 77
  25. 2016Highway and Residual Networks learn Unrolled Iterative Estimationcited by 52
  26. 2017BPEmb: Tokenization-free Pre-trained Subword Embeddings in 275 Languagescited by 52
  27. 2022Datasheet for the Pilecited by 43
  28. 2021What if This Modified That? Syntactic Interventions via Counterfactual Embeddingscited by 39
  29. 2016Logical Inductioncited by 16
  30. 2019A Comparison of Semantic Similarity Methods for Maximum Human Interpretabilitycited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 44; 2024: 112; 2025: 129; 2026: 338.

623 papers.

Sort:

By area:

  1. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  2. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  3. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 319
  4. 2023Function Vectors in Large Language Modelscited by 308
  5. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 281
  6. 2023LEACE: Perfect linear concept erasure in closed formcited by 267
  7. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 259
  8. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 235
  9. 2024The Unreasonable Ineffectiveness of the Deeper Layerscited by 226
  10. 2024Not All Language Model Features Are One-Dimensionally Linearcited by 212
  11. 2023How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelcited by 202
  12. 2023Linearity of Relation Decoding in Transformer Language Modelscited by 200
38 more
  1. 2023Localizing Model Behavior with Path Patchingcited by 195
  2. 2024Do Large Language Models Latently Perform Multi-Hop Reasoning?cited by 169
  3. 2023Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attackscited by 158
  4. 2023The Hydra Effect: Emergent Self-repair in Language Model Computationscited by 156
  5. 2023Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchillacited by 149
  6. 2023Emergent and Predictable Memorization in Large Language Modelscited by 148
  7. 2024The Remarkable Robustness of LLMs: Stages of Inference?cited by 146
  8. 2023Inspecting and Editing Knowledge Representations in Language Modelscited by 141
  9. 2024LaCo: Large Language Model Pruning via Layer Collapsecited by 126
  10. 2023Future Lens: Anticipating Subsequent Tokens from a Single Hidden Statecited by 121
  11. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 118
  12. 2024Rethinking Interpretability in the Era of Large Language Modelscited by 111
  13. 2024Interpreting and Editing Vision-Language Representations to Mitigate Hallucinationscited by 104
  14. 2024Universal Neurons in GPT2 Language Modelscited by 102
  15. 2024How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden Statescited by 98
  16. 2024Representation Noising: A Defence Mechanism Against Harmful Finetuningcited by 96
  17. 2023Jump to Conclusions: Short-Cutting Transformers With Linear Transformationscited by 90
  18. 2023A Mechanistic Interpretation of Arithmetic Reasoning in Language Models using Causal Mediation Analysiscited by 89
  19. 2026Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Modelscited by 86
  20. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 85
  21. 2025OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metricscited by 83
  22. 2023Overthinking the Truth: Understanding how Language Models Process False Demonstrationscited by 81
  23. 2024Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalizationcited by 79
  24. 2023Successor Heads: Recurring, Interpretable Attention Heads In The Wildcited by 77
  25. 2023Copy Suppression: Comprehensively Understanding an Attention Headcited by 76
  26. 2023The MiniPile Challenge for Data-Efficient Language Modelscited by 75
  27. 2026Verbalizable Representations Form a Global Workspace in Language Modelscited by 75
  28. 2024Get my drift? Catching LLM Task Drift with Activation Deltascited by 72
  29. 2024The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalitiescited by 69
  30. 2023Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Modelscited by 67
  31. 2024SelfIE: Self-Interpretation of Large Language Model Embeddingscited by 67
  32. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 66
  33. 2023HI-TOM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Modelscited by 63
  34. 2025Rethinking LLM Unlearning Objectives: A Gradient Perspective and Go Beyondcited by 61
  35. 2024Spectral Filters, Dark Signals, and Attention Sinkscited by 56
  36. 2024Do LLMs Know about Hallucination? An Empirical Investigation of LLM's Hidden Statescited by 52
  37. 2024The Geometry of Concepts: Sparse Autoencoder Feature Structurecited by 52
  38. 2025Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coincited by 52

50 most cited of 623 All 623 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Thinking into the Future: Latent Lookahead Training for Transformers
  2. 2025Closed-Loop Transformers: Autoregressive Modeling as Iterative Latent Equilibrium
  3. 2025Transformers Don't Need LayerNorm at Inference Time: Scaling LayerNorm Removal to GPT-2 XL and the Implications for Mechanistic Interpretability
  4. 2026Learn from your own latents and not from tokens: A sample-complexity theory
  5. 2026Reasoning with Latent Tokens in Diffusion Language Models
  6. 2026At the Edge of Understanding: Sparse Autoencoders Trace The Limits of Transformer Generalization
  7. 2026SNLP: Layer-Parallel Inference via Structured Newton Corrections
  8. 2026Score × Decoder: A Unified View of Unsupervised Inference-Time Scaling for Hallucination Mitigation