# Language Model Meets Prototypes: Towards Interpretable Text Classification Models through Prototypical Networks

arXiv: https://arxiv.org/abs/2412.03761 · HTML: https://mlsanity.com/paper/2412.03761/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 9 references that are in the map.
- First posted on arXiv 4 Dec 2024.

## Builds on (most cited first, 5 of 5)

- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictions](https://mlsanity.com/paper/1710.04806/) (2017)
- [Interpretable and Steerable Sequence Learning via Prototypes](https://mlsanity.com/paper/1907.09728/) (2019)
- [GAProtoNet: A Multi-head Graph Attention-based Prototypical Network for Interpretable Text Classification](https://mlsanity.com/paper/2409.13312/) (2024)

## Built on by (most cited first, 5 of 5)

- [GridRoute: A Benchmark for LLM-Based Route Planning with Cardinal Movement in Grid Environments](https://mlsanity.com/paper/2505.24306/) (2025)
- [Enhancing Nursing and Elderly Care with Large Language Models: An AI-Driven Framework](https://mlsanity.com/paper/2412.09946/) (2024)
- [A Transformer and Prototype-based Interpretable Model for Contextual Sarcasm Detection](https://mlsanity.com/paper/2503.11838/) (2025)
- [Additive Large Language Models for Semi-Structured Text](https://mlsanity.com/paper/2511.11922/) (2025)
- [Label Forensics: Interpreting Hard Labels in Black-Box Text Classifier](https://mlsanity.com/paper/2512.01514/) (2025)

## Abstract (arXiv)

Pretrained transformer-based Language Models (LMs) are well-known for their ability to achieve significant improvement on NLP tasks, but their black-box nature, which leads to a lack of interpretability, has been a major concern. My dissertation focuses on developing intrinsically interpretable models when using LMs as encoders while maintaining their superior performance via prototypical networks. I initiated my research by investigating enhancements in performance for interpretable models of sarcasm detection. My proposed approach focuses on capturing sentiment incongruity to enhance accuracy while offering instance-based explanations for the classification decisions. Later, I developed a novel white-box multi-head graph attention-based prototype network designed to explain the decisions of text classification models without sacrificing the accuracy of the original black-box LMs. In addition, I am working on extending the attention-based prototype network with contrastive learning to redesign an interpretable graph neural network, aiming to enhance both the interpretability and performance of the model in document classification.