mlsanityData to 14 Oct 2026

DecoderLens: Layerwise Interpretation of Encoder-Decoder Transformers

arXiv 2310.03686 ↗First posted 5 Oct 2023cs.CLcited by 22

Not on the map. Why?

Abstract (arXiv)

In recent years, many interpretability methods have been proposed to help interpret the internal states of Transformer-models, at different levels of precision and complexity. Here, to analyze encoder-decoder Transformers, we propose a simple, new method: DecoderLens. Inspired by the LogitLens (for decoder-only Transformers), this method involves allowing the decoder to cross-attend representations of intermediate encoder layers instead of using the final encoder output, as is normally done in encoder-decoder models. The method thus maps previously uninterpretable vector representations to human-interpretable sequences of words or symbols. We report results from the DecoderLens applied to models trained on question answering, logical reasoning, speech recognition and machine translation. The DecoderLens reveals several specific subtasks that are solved at low or intermediate layers, shedding new light on the information flow inside the encoder component of this important class of models.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 46 of its 73 references that are in the map. First posted on arXiv 5 Oct 2023.

Builds on

46 of 73 references.

References · 46

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  3. 2016Layer Normalizationcited by 5,669
  4. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  5. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  6. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  7. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  8. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  9. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  10. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  11. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
  12. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
34 more
  1. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 723
  2. 2018Universal Transformerscited by 647
  3. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  4. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 599
  5. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  6. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 532
  7. 2021Causal Abstractions of Neural Networkscited by 422
  8. 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 332
  9. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 319
  10. 2022Confident Adaptive Language Modelingcited by 310
  11. 2020BERT Loses Patience: Fast and Robust Inference with Early Exitcited by 273
  12. 2020FastBERT: a Self-distilling BERT with Adaptive Inference Timecited by 226
  13. 2018Dissecting Contextual Word Embeddings: Architecture and Representationcited by 146
  14. 2022Analyzing Transformers in Embedding Spacecited by 132
  15. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 130
  16. 2020The Right Tool for the Job: Matching Model and Instance Complexitiescited by 126
  17. 2021The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformerscited by 117
  18. 2021Encoder Fusion Network with Co-Attention Embedding for Referring Image Segmentationcited by 111
  19. 2023Language Models Implement Simple Word2Vec-style Vector Arithmeticcited by 98
  20. 2023Jump to Conclusions: Short-Cutting Transformers With Linear Transformationscited by 90
  21. 2023Overthinking the Truth: Understanding how Language Models Process False Demonstrationscited by 81
  22. 2017Residual Connections Encourage Iterative Inferencecited by 77
  23. 2020CoVoST 2 and Massively Multilingual Speech-to-Text Translationcited by 74
  24. 2020Teaching Temporal Logics to Neural Networkscited by 51
  25. 2022Towards Opening the Black Box of Neural Machine Translation: Source and Target Interpretations of the Transformercited by 45
  26. 2023Quantifying Context Mixing in Transformerscited by 43
  27. 2021The Grammar-Learning Trajectories of Neural Language Modelscited by 36
  28. 2021Language Models Use Monotonicity to Assess NPI Licensingcited by 31
  29. 2020Understanding and Improving Encoder Layer Fusion in Sequence-to-Sequence Learningcited by 26
  30. 2018Exploiting Deep Representations for Neural Machine Translationcited by 23
  31. 2021Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMTcited by 21
  32. 2022A Song of (Dis)agreement: Evaluating the Evaluation of Explainable Artificial Intelligence in Natural Language Processingcited by 13
  33. 2020Evaluating Attribution Methods using White-Box LSTMscited by 8
  34. 2023Feature Interactions Reveal Linguistic Structure in Language Modelscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 9; 2025: 7; 2026: 5.

22 papers.

Sort:

By area:

  1. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  2. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 235
  3. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 85
  4. 2023EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelismcited by 69
  5. 2024A Concept-Based Explainability Framework for Large Multimodal Modelscited by 39
  6. 2024Locate-then-edit for Multi-hop Factual Recall under Knowledge Editingcited by 37
  7. 2024Calibrating Reasoning in Language Models with Internal Consistencycited by 28
  8. 2024Human-like Linguistic Biases in Neural Speech Models: Phonetic Categorization and Phonotactic Constraints in Wav2Vec2.0cited by 19
  9. 2025Analyzing Finetuning Representation Shift for Multimodal LLMs Steeringcited by 16
  10. 2025How Do LLMs Use Their Depth?cited by 16
  11. 2025Discovering and Steering Interpretable Concepts in Large Generative Music Modelscited by 13
  12. 2025Do Activation Verbalization Methods Convey Privileged Information?cited by 10
10 more
  1. 2025Unboxing the Black Box: Mechanistic Interpretability for Algorithmic Understanding of Neural Networkscited by 6
  2. 2024Interpretability for Time Series Transformers using A Concept Bottleneck Frameworkcited by 4
  3. 2025Languages in Whisper-Style Speech Encoders Align Both Phonetically and Semanticallycited by 3
  4. 2026The Long Delay to Arithmetic Generalization: When Learned Representations Outrun Behaviorcited by 3
  5. 2025Elucidating Subspace Perturbation in Zeroth-Order Optimization: Theory and Practice at Scalecited by 2
  6. 2024Universal Response and Emergence of Induction in LLMscited by 1
  7. 2026Positional versus Symbolic Attention Heads: Learning Dynamics, RoPE Geometry, and Length Generalizationcited by 1
  8. 2026When Less Is More? Diagnosing ASR Predictions in Sardinian via Layer-Wise Decodingcited by 0
  9. 2026Evaluating Explainable AI Attribution Methods in Neural Machine Translation via Attention-Guided Knowledge Distillationcited by 0
  10. 2026Scaling Attention Head Analysis via Gradient-Based Attribution in Context-Aware Machine Translationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Unsupervised decoding of encoded reasoning using language model interpretability
  2. 2025Predictive Concept Decoders: Training Scalable End-to-End Interpretability Assistants
  3. 2025Towards Interpretability Without Sacrifice: Faithful Dense Layer Decomposition with Mixture of Decoders
  4. 2024Efficient Encoder-Decoder Transformer Decoding for Decomposable Tasks
  5. 2023Task-agnostic Distillation of Encoder-Decoder Language Models
  6. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models
  7. 2024Mechanistic Interpretability of Binary and Ternary Transformers
  8. 2026Sparse Transformer: a Controlled Testbed for Mechanistic Interpretability