# More Identifiable yet Equally Performant Transformers for Text Classification

arXiv: https://arxiv.org/abs/2106.01269 · HTML: https://mlsanity.com/paper/2106.01269/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 15 references that are in the map.
- First posted on arXiv 2 Jun 2021.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [Attention is not not Explanation](https://mlsanity.com/paper/1908.04626/) (2019)
- [On Identifiability in Transformers](https://mlsanity.com/paper/1908.04211/) (2019)

## Built on by (most cited first, 3 of 3)

- [Understanding the Failure of Batch Normalization for Transformers in NLP](https://mlsanity.com/paper/2210.05153/) (2022)
- [TegFormer: Topic-to-Essay Generation with Good Topic Coverage and High Text Coherence](https://mlsanity.com/paper/2212.13456/) (2022)
- [Unlocking Insights into Business Trajectories with Transformer-based Spatio-temporal Data Analysis](https://mlsanity.com/paper/2306.10034/) (2023)

## Abstract (arXiv)

Interpretability is an important aspect of the trustworthiness of a model's predictions. Transformer's predictions are widely explained by the attention weights, i.e., a probability distribution generated at its self-attention unit (head). Current empirical studies provide shreds of evidence that attention weights are not explanations by proving that they are not unique. A recent study showed theoretical justifications to this observation by proving the non-identifiability of attention weights. For a given input to a head and its output, if the attention weights generated in it are unique, we call the weights identifiable. In this work, we provide deeper theoretical analysis and empirical observations on the identifiability of attention weights. Ignored in the previous works, we find the attention weights are more identifiable than we currently perceive by uncovering the hidden role of the key vector. However, the weights are still prone to be non-unique attentions that make them unfit for interpretation. To tackle this issue, we provide a variant of the encoder layer that decouples the relationship between key and value vector and provides identifiable weights up to the desired length of the input. We prove the applicability of such variations by providing empirical justifications on varied text classification tasks. The implementations are available at https://github.com/declare-lab/identifiable-transformers.