# Corpus-level and Concept-based Explanations for Interpretable Document Classification

arXiv: https://arxiv.org/abs/2004.13003 · HTML: https://mlsanity.com/paper/2004.13003/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Apr 2020.

## Built on by (most cited first, 4 of 4)

- [Human-Centered Concept Explanations for Neural Networks](https://mlsanity.com/paper/2202.12451/) (2022)
- [Global Concept Explanations for Graphs by Contrastive Learning](https://mlsanity.com/paper/2404.16532/) (2024)
- [Towards eXplainable AI for Mobility Data Science](https://mlsanity.com/paper/2307.08461/) (2023)
- [Variational Language Concepts for Interpreting Foundation Language Models](https://mlsanity.com/paper/2410.03964/) (2024)

## Abstract (arXiv)

Using attention weights to identify information that is important for models' decision-making is a popular approach to interpret attention-based neural networks. This is commonly realized in practice through the generation of a heat-map for every single document based on attention weights. However, this interpretation method is fragile, and easy to find contradictory examples. In this paper, we propose a corpus-level explanation approach, which aims to capture causal relationships between keywords and model predictions via learning the importance of keywords for predicted labels across a training corpus based on attention weights. Based on this idea, we further propose a concept-based explanation method that can automatically learn higher-level concepts and their importance to model prediction tasks. Our concept-based explanation method is built upon a novel Abstraction-Aggregation Network, which can automatically cluster important keywords during an end-to-end training process. We apply these methods to the document classification task and show that they are powerful in extracting semantically meaningful keywords and concepts. Our consistency analysis results based on an attention-based Na"ive Bayes classifier also demonstrate these keywords and concepts are important for model predictions.