# An Attention Matrix for Every Decision: Faithfulness-based Arbitration Among Multiple Attention-Based Interpretations of Transformers in Text Classification

arXiv: https://arxiv.org/abs/2209.10876 · HTML: https://mlsanity.com/paper/2209.10876/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 44 references that are in the map.
- First posted on arXiv 22 Sep 2022.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [Quantifying Attention Flow in Transformers](https://mlsanity.com/paper/2005.00928/) (2020)
- [Attention is not not Explanation](https://mlsanity.com/paper/1908.04626/) (2019)
- [Transformer Interpretability Beyond Attention Visualization](https://mlsanity.com/paper/2012.09838/) (2020)
- [Carbon Emissions and Large Neural Network Training](https://mlsanity.com/paper/2104.10350/) (2021)
- [ERASER: A Benchmark to Evaluate Rationalized NLP Models](https://mlsanity.com/paper/1911.03429/) (2019)
- [Towards Robust Interpretability with Self-Explaining Neural Networks](https://mlsanity.com/paper/1806.07538/) (2018)
- [e-SNLI: Natural Language Inference with Natural Language Explanations](https://mlsanity.com/paper/1812.01193/) (2018)
- [HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection](https://mlsanity.com/paper/2012.10289/) (2020)
- [Revealing the Dark Secrets of BERT](https://mlsanity.com/paper/1908.08593/) (2019)
- [A Multiscale Visualization of Attention in the Transformer Model](https://mlsanity.com/paper/1906.05714/) (2019)
- [Explainable Prediction of Medical Codes from Clinical Text](https://mlsanity.com/paper/1802.05695/) (2018)
- [The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?](https://mlsanity.com/paper/2010.05607/) (2020)
- [On Identifiability in Transformers](https://mlsanity.com/paper/1908.04211/) (2019)
- [Tree Transformer: Integrating Tree Structures into Self-Attention](https://mlsanity.com/paper/1909.06639/) (2019)
- [ETHOS: an Online Hate Speech Detection Dataset](https://mlsanity.com/paper/2006.08328/) (2020)
- [The Promise and Peril of Human Evaluation for Model Interpretability](https://mlsanity.com/paper/1711.07414/) (2017)
- [Rethinking Attention-Model Explainability through Faithfulness Violation Test](https://mlsanity.com/paper/2201.12114/) (2022)
- [A Comparative Study of Faithfulness Metrics for Model Interpretability Methods](https://mlsanity.com/paper/2204.05514/) (2022)
- [Human-grounded Evaluations of Explanation Methods for Text Classification](https://mlsanity.com/paper/1908.11355/) (2019)
- [Does BERT Learn as Humans Perceive? Understanding Linguistic Styles through Lexica](https://mlsanity.com/paper/2109.02738/) (2021)
- [On Attribution of Recurrent Neural Network Predictions via Additive Decomposition](https://mlsanity.com/paper/1903.11245/) (2019)
- [Thermostat: A Large Collection of NLP Model Explanations and Analysis Tools](https://mlsanity.com/paper/2108.13961/) (2021)
- [On the Granularity of Explanations in Model Agnostic NLP Interpretability](https://mlsanity.com/paper/2012.13189/) (2020)
- [LioNets: A Neural-Specific Local Interpretation Technique Exploiting Penultimate Layer Information](https://mlsanity.com/paper/2104.06057/) (2021)

## Built on by (most cited first, 6 of 6)

- [Training a Utility-based Retriever Through Shared Context Attribution for Retrieval-Augmented Language Models](https://mlsanity.com/paper/2504.00573/) (2025)
- [Attention Meets Post-hoc Interpretability: A Mathematical Perspective](https://mlsanity.com/paper/2402.03485/) (2024)
- [Multi-Layer Attention-Based Explainability via Transformers for Tabular Data](https://mlsanity.com/paper/2302.14278/) (2023)
- [Faithful and Robust Local Interpretability for Textual Predictions](https://mlsanity.com/paper/2311.01605/) (2023)
- [Combining physics education and machine learning research to measure evidence of students' mechanistic sensemaking](https://mlsanity.com/paper/2503.15638/) (2025)
- [There is More to Attention: Statistical Filtering Enhances Explanations in Vision Transformers](https://mlsanity.com/paper/2510.06070/) (2025)

## Abstract (arXiv)

Transformers are widely used in natural language processing, where they consistently achieve state-of-the-art performance. This is mainly due to their attention-based architecture, which allows them to model rich linguistic relations between (sub)words. However, transformers are difficult to interpret. Being able to provide reasoning for its decisions is an important property for a model in domains where human lives are affected. With transformers finding wide use in such fields, the need for interpretability techniques tailored to them arises. We propose a new technique that selects the most faithful attention-based interpretation among the several ones that can be obtained by combining different head, layer and matrix operations. In addition, two variations are introduced towards (i) reducing the computational complexity, thus being faster and friendlier to the environment, and (ii) enhancing the performance in multi-label data. We further propose a new faithfulness metric that is more suitable for transformer models and exhibits high correlation with the area under the precision-recall curve based on ground truth rationales. We validate the utility of our contributions with a series of quantitative and qualitative experiments on seven datasets.