# From Clustering to Cluster Explanations via Neural Networks

arXiv: https://arxiv.org/abs/1906.07633 · HTML: https://mlsanity.com/paper/1906.07633/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2019.

## Built on by (most cited first, 30 of 34)

- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [A Comprehensive Taxonomy for Explainable Artificial Intelligence: A Systematic Survey of Surveys on Methods and Concepts](https://mlsanity.com/paper/2105.07190/) (2021)
- [This looks more like that: Enhancing Self-Explaining Models by Prototypical Relevance Propagation](https://mlsanity.com/paper/2108.12204/) (2021)
- [Toward Explainable AI for Regression Models](https://mlsanity.com/paper/2112.11407/) (2021)
- [Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspaces](https://mlsanity.com/paper/2212.14855/) (2022)
- [Interpretable Deep Clustering for Tabular Data](https://mlsanity.com/paper/2306.04785/) (2023)
- [Shallow decision trees for explainable k-means clustering](https://mlsanity.com/paper/2112.14718/) (2021)
- [Near-Optimal Explainable k-Means for All Dimensions](https://mlsanity.com/paper/2106.15566/) (2021)
- [Interpretable Clustering via Multi-Polytope Machines](https://mlsanity.com/paper/2112.05653/) (2021)
- [Explaining Predictive Uncertainty by Exposing Second-Order Effects](https://mlsanity.com/paper/2401.17441/) (2024)
- [Capturing Polysemanticity with PRISM: A Multi-Concept Feature Description Framework](https://mlsanity.com/paper/2506.15538/) (2025)
- [Preemptively Pruning Clever-Hans Strategies in Deep Neural Networks](https://mlsanity.com/paper/2304.05727/) (2023)
- [Cluster Explanation via Polyhedral Descriptions](https://mlsanity.com/paper/2210.08798/) (2022)
- [Explainable AI needs formalization](https://mlsanity.com/paper/2409.14590/) (2024)
- [Position: We Need An Algorithmic Understanding of Generative AI](https://mlsanity.com/paper/2507.07544/) (2025)
- [Self-Supervised Training with Autoencoders for Visual Anomaly Detection](https://mlsanity.com/paper/2206.11723/) (2022)
- [NeurCAM: Interpretable Neural Clustering via Additive Models](https://mlsanity.com/paper/2408.13361/) (2024)
- [Understanding Text Classification Data and Models Using Aggregated Input Salience](https://mlsanity.com/paper/2211.05485/) (2022)
- [Explainable Graph Spectral Clustering of Text Documents](https://mlsanity.com/paper/2308.00504/) (2023)
- [LLMs grasp morality in concept](https://mlsanity.com/paper/2311.02294/) (2023)
- [Multi-Depth Concept Extraction for Post-Hoc Vision Encoder Explanation](https://mlsanity.com/paper/2411.19700/) (2024)
- [Wasserstein Distances Made Explainable: Insights Into Dataset Shifts and Transport Phenomena](https://mlsanity.com/paper/2505.06123/) (2025)
- [Explaining deep learning for ECG using time-localized clusters](https://mlsanity.com/paper/2509.15198/) (2025)
- [Normalized Relevance Measure as a Unifying Framework to Explain Neural Network Latent Structures](https://mlsanity.com/paper/2606.00557/) (2026)
- [Feature graphs for interpretable unsupervised tree ensembles: centrality, interaction, and application in disease subtyping](https://mlsanity.com/paper/2404.17886/) (2024)
- [AnomalyAID: Reliable Interpretation for Semi-supervised Network Anomaly Detection](https://mlsanity.com/paper/2411.11293/) (2024)
- [Cat, Rat, Meow: On the Alignment of Language Model and Human Term-Similarity Judgments](https://mlsanity.com/paper/2504.07965/) (2025)
- [Unveiling 3D Ocean Biogeochemical Provinces in the North Atlantic: A Systematic Comparison and Validation of Clustering Methods](https://mlsanity.com/paper/2504.18181/) (2025)
- [From Local to Global to Mechanistic: An iERF-Centered Unified Framework for Interpreting Vision Models](https://mlsanity.com/paper/2605.00474/) (2026)
- [Leveraging Activations for Superpixel Explanations](https://mlsanity.com/paper/2406.04933/) (2024)

## Abstract (arXiv)

A recent trend in machine learning has been to enrich learned models with the ability to explain their own predictions. The emerging field of Explainable AI (XAI) has so far mainly focused on supervised learning, in particular, deep neural network classifiers. In many practical problems however, label information is not given and the goal is instead to discover the underlying structure of the data, for example, its clusters. While powerful methods exist for extracting the cluster structure in data, they typically do not answer the question why a certain data point has been assigned to a given cluster. We propose a new framework that can, for the first time, explain cluster assignments in terms of input features in an efficient and reliable manner. It is based on the novel insight that clustering models can be rewritten as neural networks - or 'neuralized'. Cluster predictions of the obtained networks can then be quickly and accurately attributed to the input features. Several showcases demonstrate the ability of our method to assess the quality of learned clusters and to extract novel insights from the analyzed data and representations.