mlsanityData to 14 Oct 2026

Decoding Layer Saliency in Language Transformers

arXiv 2308.05219 ↗First posted 9 Aug 2023cs.CL cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

In this paper, we introduce a strategy for identifying textual saliency in large-scale language models applied to classification tasks. In visual networks where saliency is more well-studied, saliency is naturally localized through the convolutional layers of the network; however, the same is not true in modern transformer-stack networks used to process natural language. We adapt gradient-based saliency methods for these networks, propose a method for evaluating the degree of semantic coherence of each layer, and demonstrate consistent improvement over numerous other methods for textual saliency on multiple benchmark classification datasets. Our approach requires no additional training or access to labelled data, and is comparatively very computationally efficient.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 47 of its 64 references that are in the map. First posted on arXiv 9 Aug 2023.

Builds on

47 of 64 references.

References · 47

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  5. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  6. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  7. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  8. 2015Learning Deep Features for Discriminative Localizationcited by 2,436
  9. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  10. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  11. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
  12. 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
35 more
  1. 2017SmoothGrad: removing noise by adding noisecited by 1,109
  2. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
  3. 2014Striving for Simplicity: The All Convolutional Netcited by 990
  4. 2019Attention is not Explanationcited by 929
  5. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 856
  6. 2019Are Sixteen Heads Really Better than One?cited by 816
  7. 2020A Primer in BERTology: What we know about how BERT workscited by 815
  8. 2019Attention is not not Explanationcited by 636
  9. 2020Transformer Interpretability Beyond Attention Visualizationcited by 627
  10. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 612
  11. 2019Is Attention Interpretable?cited by 389
  12. 2015Learning Deconvolution Network for Semantic Segmentationcited by 365
  13. 2016From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classificationcited by 341
  14. 2016Top-down Neural Attention by Excitation Backpropcited by 321
  15. 2021Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformerscited by 320
  16. 2019Revealing the Dark Secrets of BERTcited by 301
  17. 2019A Multiscale Visualization of Attention in the Transformer Modelcited by 292
  18. 2016Understanding Neural Networks through Representation Erasurecited by 224
  19. 2019Transformer Dissection: A Unified Understanding of Transformer's Attention via the Lens of Kernelcited by 213
  20. 2020Self-Attention Attribution: Interpreting Information Interactions Inside Transformercited by 125
  21. 2020The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?cited by 122
  22. 2019Open Sesame: Getting Inside BERT's Linguistic Knowledgecited by 120
  23. 2019Smooth Grad-CAM++: An Enhanced Inference Level Visualization Technique for Deep Convolutional Neural Network Modelscited by 118
  24. 2019Learning to Deceive with Attention-Based Explanationscited by 111
  25. 2017Explaining Recurrent Neural Network Predictions in Sentiment Analysiscited by 88
  26. 2021The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanationscited by 87
  27. 2019AllenNLP Interpret: A Framework for Explaining Predictions of NLP Modelscited by 74
  28. 2019Attention Interpretability Across NLP Taskscited by 71
  29. 2018Seq2Seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Modelscited by 69
  30. 2020How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Maskingcited by 63
  31. 2021On the Expressive Power of Self-Attention Matricescited by 47
  32. 2018Understanding Individual Decisions of CNNs via Contrastive Backpropagationcited by 40
  33. 2016Representation of linguistic form and function in recurrent neural networkscited by 39
  34. 2019Generating Token-Level Explanations for Natural Language Inferencecited by 28
  35. 2021Thermostat: A Large Collection of NLP Model Explanations and Analysis Toolscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 27
  2. 2025Head Pursuit: Probing Attention Specialization in Multimodal Transformerscited by 17
  3. 2026Layer-wise Positional Bias in Short-Context Language Modelingcited by 1
  4. 2026Explainable AI: Context-Aware Layer-Wise Integrated Gradients for Explaining Transformer Modelscited by 1
  5. 2025NLP4Neuro: Sequence-to-sequence learning for neural population decodingcited by 0