# GAProtoNet: A Multi-head Graph Attention-based Prototypical Network for Interpretable Text Classification

arXiv: https://arxiv.org/abs/2409.13312 · HTML: https://mlsanity.com/paper/2409.13312/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 27 references that are in the map.
- First posted on arXiv 20 Sep 2024.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Universal Sentence Encoder](https://mlsanity.com/paper/1803.11175/) (2018)
- [KGAT: Knowledge Graph Attention Network for Recommendation](https://mlsanity.com/paper/1905.07854/) (2019)
- [Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictions](https://mlsanity.com/paper/1710.04806/) (2017)
- [Interpretable and Steerable Sequence Learning via Prototypes](https://mlsanity.com/paper/1907.09728/) (2019)
- [ProtoTEx: Explaining Model Decisions with Prototype Tensors](https://mlsanity.com/paper/2204.05426/) (2022)

## Built on by (most cited first, 4 of 4)

- [Interpretability Can Be Actionable](https://mlsanity.com/paper/2605.11161/) (2026)
- [Language Model Meets Prototypes: Towards Interpretable Text Classification Models through Prototypical Networks](https://mlsanity.com/paper/2412.03761/) (2024)
- [Enhancing Nursing and Elderly Care with Large Language Models: An AI-Driven Framework](https://mlsanity.com/paper/2412.09946/) (2024)
- [ProtoSiTex: Learning Semi-Interpretable Prototypes for Multi-label Text Classification](https://mlsanity.com/paper/2510.12534/) (2025)

## Abstract (arXiv)

Pretrained transformer-based Language Models (LMs) are well-known for their ability to achieve significant improvement on text classification tasks with their powerful word embeddings, but their black-box nature, which leads to a lack of interpretability, has been a major concern. In this work, we introduce GAProtoNet, a novel white-box Multi-head Graph Attention-based Prototypical Network designed to explain the decisions of text classification models built with LM encoders. In our approach, the input vector and prototypes are regarded as nodes within a graph, and we utilize multi-head graph attention to selectively construct edges between the input node and prototype nodes to learn an interpretable prototypical representation. During inference, the model makes decisions based on a linear combination of activated prototypes weighted by the attention score assigned for each prototype, allowing its choices to be transparently explained by the attention weights and the prototypes projected into the closest matching training examples. Experiments on multiple public datasets show our approach achieves superior results without sacrificing the accuracy of the original black-box LMs. We also compare with four alternative prototypical network variations and our approach achieves the best accuracy and F1 among all. Our case study and visualization of prototype clusters also demonstrate the efficiency in explaining the decisions of black-box models built with LMs.