# Understanding Multi-Head Attention in Abstractive Summarization

arXiv: https://arxiv.org/abs/1911.03898 · HTML: https://mlsanity.com/paper/1911.03898/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Nov 2019.

## Built on by (most cited first, 12 of 12)

- [Pre-Trained Models: Past, Present and Future](https://mlsanity.com/paper/2106.07139/) (2021)
- [Neurons in Large Language Models: Dead, N-gram, Positional](https://mlsanity.com/paper/2309.04827/) (2023)
- [Order in the Court: Explainable AI Methods Prone to Disagreement](https://mlsanity.com/paper/2105.03287/) (2021)
- [Optimal ablation for interpretability](https://mlsanity.com/paper/2409.09951/) (2024)
- [A Song of (Dis)agreement: Evaluating the Evaluation of Explainable Artificial Intelligence in Natural Language Processing](https://mlsanity.com/paper/2205.04559/) (2022)
- [Transformer Explainer: Learning LLM Transformers with Interactive Visual Explanation and Experimentation](https://mlsanity.com/paper/2408.04619/) (2024)
- [HiFi: High-Information Attention Heads Hold for Parameter-Efficient Model Adaptation](https://mlsanity.com/paper/2305.04573/) (2023)
- [TgDLF2.0: Theory-guided deep-learning for electrical load forecasting via Transformer and transfer learning](https://mlsanity.com/paper/2210.02448/) (2022)
- [Source Identification in Abstractive Summarization](https://mlsanity.com/paper/2402.04677/) (2024)
- [Explanation on Pretraining Bias of Finetuned Vision Transformer](https://mlsanity.com/paper/2211.15428/) (2022)
- [CNN with large memory layers](https://mlsanity.com/paper/2101.11685/) (2021)
- [Extracting Interpretable Local and Global Representations from Attention on Time Series](https://mlsanity.com/paper/2312.11466/) (2023)

## Abstract (arXiv)

Attention mechanisms in deep learning architectures have often been used as a means of transparency and, as such, to shed light on the inner workings of the architectures. Recently, there has been a growing interest in whether or not this assumption is correct. In this paper we investigate the interpretability of multi-head attention in abstractive summarization, a sequence-to-sequence task for which attention does not have an intuitive alignment role, such as in machine translation. We first introduce three metrics to gain insight in the focus of attention heads and observe that these heads specialize towards relative positions, specific part-of-speech tags, and named entities. However, we also find that ablating and pruning these heads does not lead to a significant drop in performance, indicating redundancy. By replacing the softmax activation functions with sparsemax activation functions, we find that attention heads behave seemingly more transparent: we can ablate fewer heads and heads score higher on our interpretability metrics. However, if we apply pruning to the sparsemax model we find that we can prune even more heads, raising the question whether enforced sparsity actually improves transparency. Finally, we find that relative positions heads seem integral to summarization performance and persistently remain after pruning.