Data to 7 Oct 2026

Attention Interpretability Across NLP Tasks

arXiv 1909.11218 ↗First posted 24 Sep 2019cs.CL cs.LGcited by 71

Not on the map. Why?

Abstract (arXiv)

The attention layer in a neural network model provides insights into the model's reasoning behind its prediction, which are usually criticized for being opaque. Recently, seemingly contradictory viewpoints have emerged about the interpretability of attention weights (Jain & Wallace, 2019; Vig & Belinkov, 2019). Amid such confusion arises the need to understand attention mechanism more systematically. In this work, we attempt to fill this gap by giving a comprehensive explanation which justifies both kinds of observations (i.e., when is attention interpretable and when it is not). Through a series of experiments on diverse NLP tasks, we validate our observations and reinforce our claim of interpretability of attention through manual evaluation.

Key facts. Cited by 71 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 21; 2023: 15; 2024: 14; 2025: 6; 2026: 2.

71 papers.

Sort:

By area:

2021Post-hoc Interpretability for Neural NLP: A Surveycited by 155
2024Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Mapscited by 140
2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
2024Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilizationcited by 97
2023Attention Sorting Combats Recency Bias In Long Context Language Modelscited by 89
2021Unmasking the Mask -- Evaluating Social Biases in Masked Language Modelscited by 60
2022Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translationcited by 60
2021Approximating How Single Head Attention Learnscited by 38
2022On the Explainability of Natural Language Processing Deep Modelscited by 37
2025A Survey on Mechanistic Interpretability for Multi-Modal Foundation Modelscited by 34
2021Evaluating the Faithfulness of Importance Measures in NLP by Recursively Masking Allegedly Important Tokens and Retrainingcited by 31

38 more

2021Improving the Faithfulness of Attention-based Explanations with Task-specific Information for Text Classificationcited by 29
2024InternalInspector I2: Robust Confidence Estimation in LLMs through Internal Statescited by 29
2021Dodrio: Exploring Transformer Models with Interactive Visualizationcited by 28
2021Rationales for Sequential Predictionscited by 26
2021Counterfactual Explanations for Models of Codecited by 24
2022Learning to Scaffold: Optimizing Model Explanations for Teachingcited by 20
2021Interpreting Deep Learning Models in Natural Language Processing: A Reviewcited by 19
2022Systematic Generalization and Emergent Structures in Transformers Trained on Structured Taskscited by 19
2021Enjoy the Salience: Towards Better Transformer-based Faithful Explanations with Word Saliencecited by 16
2024Selective Prompt Anchoring for Code Generationcited by 16
2022A Song of (Dis)agreement: Evaluating the Evaluation of Explainable Artificial Intelligence in Natural Language Processingcited by 13
2023Do Large Language Models Pay Similar Attention Like Human Programmers When Generating Code?cited by 13
2022Convexifying Transformers: Improving optimization and understanding of transformer networkscited by 12
2023IMBERT: Making BERT Immune to Insertion-based Backdoor Attackscited by 12
2024Hindsight PRIORs for Reward Learning from Human Preferencescited by 12
2024From Explainable to Interpretable Deep Learning for Natural Language Processing in Healthcare: How Far from Reality?cited by 11
2025Learning to Attribute with Attentioncited by 11
2024Interpretability Needs a New Paradigmcited by 10
2022Interpretation of Black Box NLP Models: A Surveycited by 9
2023Interpreting and Exploiting Functional Specialization in Multi-Head Attention under Multi-task Learningcited by 9
2025Dynamic Attention-Guided Context Decoding for Mitigating Context Faithfulness Hallucinations in Large Language Modelscited by 9
2022More Interpretable Graph Similarity Computation via Maximum Common Subgraph Inferencecited by 8
2023Predicting Hateful Discussions on Reddit using Graph Transformer Networks and Communal Contextcited by 8
2022Silence is Sweeter Than Speech: Self-Supervised Model Using Silence to Store Speaker Informationcited by 7
2023Roles of Scaling and Instruction Tuning in Language Perception: Model vs. Human Attentioncited by 7
2023A Theoretical Insight into Attack and Defense of Gradient Leakage in Transformercited by 6
2024Recasting Generic Pretrained Vision Transformers As Object-Centric Scene Encoders For Manipulation Policiescited by 6
2022Follow-up Attention: An Empirical Study of Developer and Neural Model Code Explorationcited by 5
2023Evaluating self-attention interpretability through human-grounded experimental protocolcited by 5
2023Decoding Layer Saliency in Language Transformerscited by 5
2023Faithfulness Measurable Masked Language Modelscited by 5
2022Should attention be all we need? The epistemic and ethical implications of unification in machine learningcited by 4
2022Is Attention Interpretation? A Quantitative Assessment On Setscited by 4
2023ViT-ReciproCAM: Gradient and Attention-Free Visual Explanations for Vision Transformercited by 4
2021Using Integrated Gradients and Constituency Parse Trees to explain Linguistic Acceptability learnt by BERTcited by 3
2022Respecting Time Series Properties Makes Deep Time Series Forecasting Perfectcited by 3
2021Multinational Address Parsing: A Zero-Shot Evaluationcited by 2
2022Understanding microbiome dynamics via interpretable graph representation learningcited by 2

50 most cited of 71 All 71 (JSON)