Data to 7 Oct 2026

Self-Attention Attribution: Interpreting Information Interactions Inside Transformer

arXiv 2004.11207 ↗First posted 23 Apr 2020cs.CLcited by 124

Not on the map. Why?

Abstract (arXiv)

The great success of Transformer-based models benefits from the powerful multi-head self-attention mechanism, which learns token dependencies and encodes contextual information from the input. Prior work strives to attribute model decisions to individual input features with different saliency measures, but they fail to explain how these input features interact with each other to reach predictions. In this paper, we propose a self-attention attribution method to interpret the information interactions inside Transformer. We take BERT as an example to conduct extensive studies. Firstly, we apply self-attention attribution to identify the important attention heads, while others can be pruned with marginal performance degradation. Furthermore, we extract the most salient dependencies in each layer to construct an attribution tree, which reveals the hierarchical interactions inside Transformer. Finally, we show that the attribution results can be used as adversarial patterns to implement non-targeted attacks towards BERT.

Key facts. Cited by 124 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 24; 2023: 26; 2024: 30; 2025: 24; 2026: 6.

124 papers.

Sort:

By area:

2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
2021Knowledge Neurons in Pretrained Transformerscited by 590
2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 367
2023Explainability for Large Language Models: A Surveycited by 331
2023PMET: Precise Model Editing in a Transformercited by 170
2021On the Robustness of Vision Transformers to Adversarial Examplescited by 167
2024Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Mapscited by 140
2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
2023Large language models in healthcare and medical domain: A reviewcited by 126
2022Revealing the Dark Secrets of Masked Image Modelingcited by 117
2024Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilizationcited by 97

38 more

2024Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Modelscited by 87
2021Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Modelscited by 76
2023Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformerscited by 67
2021Adversarial Robustness Comparison of Vision Transformer and MLP-Mixer to CNNscited by 57
2022A Study of the Attention Abnormality in Trojaned BERTscited by 47
2024From Understanding to Utilization: A Survey on Explainability for Large Language Modelscited by 47
2022Rethinking Attention-Model Explainability through Faithfulness Violation Testcited by 43
2022TransPolymer: a Transformer-based language model for polymer property predictionscited by 43
2024Enhancing Descriptive Image Quality Assessment with A Large-scale Multi-modal Datasetcited by 38
2022Kformer: Knowledge Injection in Transformer Feed-Forward Layerscited by 37
2023Finding and Editing Multi-Modal Neurons in Pre-Trained Transformerscited by 36
2024IRCAN: Mitigating Knowledge Conflicts in LLM Generation via Identifying and Reweighting Context-Aware Neuronscited by 36
2025A Survey on Mechanistic Interpretability for Multi-Modal Foundation Modelscited by 34
2024Towards Understanding the Fragility of Multilingual LLMs against Fine-Tuning Attackscited by 33
2022Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectivescited by 32
2023Attention-Enhancing Backdoor Attacks Against BERT-based Modelscited by 32
2023How Does Attention Work in Vision Transformers? A Visual Analytics Attemptcited by 29
2022EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithmcited by 27
2021On Explaining Your Explanations of BERT: An Empirical Study with Sequence Classificationcited by 21
2021Distilled Dual-Encoder Model for Vision-Language Understandingcited by 21
2021Connecting Attributions and QA Model Behavior on Realistic Counterfactualscited by 20
2021Logic Traps in Evaluating Attribution Scorescited by 18
2022Improving Semantic Matching through Dependency-Enhanced Pre-trained Model with Adaptive Fusioncited by 18
2024Pay Attention and Move Better: Harnessing Attention for Interactive Motion Generation and Training-free Editingcited by 17
2024DA-MoE: Towards Dynamic Expert Allocation for Mixture-of-Experts Modelscited by 15
2021Layer-wise Pruning of Transformer Attention Heads for Efficient Language Modelingcited by 14
2024Identifying and Mitigating Social Bias Knowledge in Language Modelscited by 13
2024Dense Passage Retrieval: Is it Retrieving?cited by 12
2025CoME: An Unlearning-based Approach to Conflict-free Model Editingcited by 12
2025LayerIF: Estimating Layer Quality for Large Language Models using Influence Functionscited by 12
2021EEGDnet: Fusing Non-Local and Local Self-Similarity for 1-D EEG Signal Denoising with 2-D Transformercited by 11
2022eX-ViT: A Novel eXplainable Vision Transformer for Weakly Supervised Semantic Segmentationcited by 11
2024CFSP: An Efficient Structured Pruning Framework for LLMs with Coarse-to-Fine Activation Informationcited by 11
2024Unveiling Concept Attribution in Diffusion Modelscited by 11
2025Attention (as Discrete-Time Markov) Chainscited by 11
2021Attributing Fair Decisions with Attention Interventionscited by 10
2023SketchXAI: A First Look at Explainability for Human Sketchescited by 10
2023Instruction Position Matters in Sequence Generation with Large Language Modelscited by 10

50 most cited of 124 All 124 (JSON)