mlsanityData to 14 Oct 2026

Interactively Providing Explanations for Transformer Language Models

arXiv 2110.02058 ↗First posted 2 Sep 2021cs.CL cs.AI cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Transformer language models are state of the art in a multitude of NLP tasks. Despite these successes, their opaqueness remains problematic. Recent methods aiming to provide interpretability and explainability to black-box models primarily focus on post-hoc explanations of (sometimes spurious) input-output correlations. Instead, we emphasize using prototype networks directly incorporated into the model architecture and hence explain the reasoning process behind the network's decisions. Our architecture performs on par with several language models and, moreover, enables learning from user interactions. This not only offers a better understanding of language models but uses human capabilities to incorporate knowledge outside of the rigid range of purely data-driven approaches.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 32 of its 77 references that are in the map. First posted on arXiv 2 Sep 2021.

Builds on

32 of 77 references.

References · 32

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  6. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  7. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  8. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  9. 2017SmoothGrad: removing noise by adding noisecited by 1,109
  10. 2019Attention is not Explanationcited by 929
  11. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 856
  12. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 784
20 more
  1. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 737
  2. 2019Attention is not not Explanationcited by 636
  3. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 530
  4. 2017Right for the Right Reasons: Training Differentiable Models by Constraining their Explanationscited by 294
  5. 2018Analysis Methods in Neural Language Processing: A Surveycited by 277
  6. 2017Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictionscited by 234
  7. 2020A Diagnostic Study of Explainability Techniques for Text Classificationcited by 159
  8. 2020Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functionscited by 144
  9. 2020Making deep neural networks right for the right scientific reasons by interacting with their explanationscited by 142
  10. 2019The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectivescited by 139
  11. 2019Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Groundedcited by 128
  12. 2020The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Modelscited by 125
  13. 2019Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approachescited by 101
  14. 2020Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanationscited by 94
  15. 2020Aligning Faithful Interpretations with their Social Attributioncited by 79
  16. 2019Interpretable and Steerable Sequence Learning via Prototypescited by 72
  17. 2019Interpretable Image Recognition with Hierarchical Prototypescited by 65
  18. 2021Evaluating Saliency Methods for Neural Language Modelscited by 37
  19. 2021An Empirical Comparison of Instance Attribution Methods for NLPcited by 35
  20. 2021What's in your Head? Emergent Behaviour in Multi-Task Transformer Modelscited by 9

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 1.

4 papers.

  1. 2023Learning by Self-Explainingcited by 17
  2. 2024A Survey on Symbolic Knowledge Distillation of Large Language Modelscited by 9
  3. 2022Revision Transformers: Instructing Language Models to Change their Valuescited by 8
  4. 2023Proto-lm: A Prototypical Network-Based Framework for Built-in Interpretability in Large Language Modelscited by 7