# Attention is Not Only a Weight: Analyzing Transformers with Vector Norms

arXiv: https://arxiv.org/abs/2004.10102 · HTML: https://mlsanity.com/paper/2004.10102/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Apr 2020.

## Built on by (most cited first, 6 of 6)

- [Visualizing and Explaining Language Models](https://mlsanity.com/paper/2205.10238/) (2022)
- [Can Language Representation Models Think in Bets?](https://mlsanity.com/paper/2210.07519/) (2022)
- [Understanding Prior Bias and Choice Paralysis in Transformer-based Language Representation Models through Four Experimental Probes](https://mlsanity.com/paper/2210.01258/) (2022)
- [Reasoning Fails Where Step Flow Breaks](https://mlsanity.com/paper/2604.06695/) (2026)
- [Multi-Stream Transformers](https://mlsanity.com/paper/2107.10342/) (2021)
- [Discovering material information using hierarchical Reformer model on financial regulatory filings](https://mlsanity.com/paper/2204.05979/) (2022)

## Abstract (arXiv)

Attention is a key component of Transformers, which have recently achieved considerable success in natural language processing. Hence, attention is being extensively studied to investigate various linguistic capabilities of Transformers, focusing on analyzing the parallels between attention weights and specific linguistic phenomena. This paper shows that attention weights alone are only one of the two factors that determine the output of attention and proposes a norm-based analysis that incorporates the second factor, the norm of the transformed input vectors. The findings of our norm-based analyses of BERT and a Transformer-based neural machine translation system include the following: (i) contrary to previous studies, BERT pays poor attention to special tokens, and (ii) reasonable word alignment can be extracted from attention mechanisms of Transformer. These findings provide insights into the inner workings of Transformers.