# Telling BERT's full story: from Local Attention to Global Aggregation

arXiv: https://arxiv.org/abs/2004.05916 · HTML: https://mlsanity.com/paper/2004.05916/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Apr 2020.

## Built on by (most cited first, 15 of 15)

- [Towards Faithful Model Explanation in NLP: A Survey](https://mlsanity.com/paper/2209.11326/) (2022)
- [Measuring the Mixing of Contextual Information in the Transformer](https://mlsanity.com/paper/2203.04212/) (2022)
- [GlobEnc: Quantifying Global Token Attribution by Incorporating the Whole Encoder Layer in Transformers](https://mlsanity.com/paper/2205.03286/) (2022)
- [Quantifying Context Mixing in Transformers](https://mlsanity.com/paper/2301.12971/) (2023)
- [DecompX: Explaining Transformers Decisions by Propagating Token Decomposition](https://mlsanity.com/paper/2306.02873/) (2023)
- [AD-KD: Attribution-Driven Knowledge Distillation for Language Model Compression](https://mlsanity.com/paper/2305.10010/) (2023)
- [Towards BERT-based Automatic ICD Coding: Limitations and Opportunities](https://mlsanity.com/paper/2104.06709/) (2021)
- [AdapLeR: Speeding up Inference by Adaptive Length Reduction](https://mlsanity.com/paper/2203.08991/) (2022)
- [Of Non-Linearity and Commutativity in BERT](https://mlsanity.com/paper/2101.04547/) (2021)
- [Enjoy the Salience: Towards Better Transformer-based Faithful Explanations with Word Salience](https://mlsanity.com/paper/2108.13759/) (2021)
- [On the Distribution, Sparsity, and Inference-time Quantization of Attention Values in Transformers](https://mlsanity.com/paper/2106.01335/) (2021)
- [Insights Into the Inner Workings of Transformer Models for Protein Function Prediction](https://mlsanity.com/paper/2309.03631/) (2023)
- [Contrast-CAT: Contrasting Activations for Enhanced Interpretability in Transformer-based Text Classifiers](https://mlsanity.com/paper/2507.21186/) (2025)
- [Layer-wise Positional Bias in Short-Context Language Modeling](https://mlsanity.com/paper/2601.04098/) (2026)
- [BOSCH: Black-Box Binary Optimization for Short-Context Attention-Head Selection in LLMs](https://mlsanity.com/paper/2604.05942/) (2026)

## Abstract (arXiv)

We take a deep look into the behavior of self-attention heads in the transformer architecture. In light of recent work discouraging the use of attention distributions for explaining a model's behavior, we show that attention distributions can nevertheless provide insights into the local behavior of attention heads. This way, we propose a distinction between local patterns revealed by attention and global patterns that refer back to the input, and analyze BERT from both angles. We use gradient attribution to analyze how the output of an attention attention head depends on the input tokens, effectively extending the local attention-based analysis to account for the mixing of information throughout the transformer layers. We find that there is a significant discrepancy between attention and attribution distributions, caused by the mixing of context inside the model. We quantify this discrepancy and observe that interestingly, there are some patterns that persist across all layers despite the mixing.