# Explaining Latent Representations with a Corpus of Examples

arXiv: https://arxiv.org/abs/2110.15355 · HTML: https://mlsanity.com/paper/2110.15355/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 43 references that are in the map.
- First posted on arXiv 28 Oct 2021.

## Builds on (most cited first, 25 of 25)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Data Shapley: Equitable Valuation of Data for Machine Learning](https://mlsanity.com/paper/1904.02868/) (2019)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI](https://mlsanity.com/paper/1907.07374/) (2019)
- [Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negatives](https://mlsanity.com/paper/1802.07623/) (2018)
- [Understanding Deep Networks via Extremal Perturbations and Smooth Masks](https://mlsanity.com/paper/1910.08485/) (2019)
- [Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey](https://mlsanity.com/paper/2006.11371/) (2020)
- [EMNIST: an extension of MNIST to handwritten letters](https://mlsanity.com/paper/1702.05373/) (2017)
- [Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning](https://mlsanity.com/paper/1803.04765/) (2018)
- [Representer Point Selection for Explaining Deep Neural Networks](https://mlsanity.com/paper/1811.09720/) (2018)
- [Deep Subspace Clustering Networks](https://mlsanity.com/paper/1709.02508/) (2017)
- [The effectiveness of feature attribution methods and its correlation with automatic evaluation scores](https://mlsanity.com/paper/2105.14944/) (2021)
- [A Distributional Framework for Data Valuation](https://mlsanity.com/paper/2002.12334/) (2020)
- [The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classification](https://mlsanity.com/paper/1503.01161/) (2015)
- [Explaining Time Series Predictions with Dynamic Masks](https://mlsanity.com/paper/2106.05303/) (2021)
- [Robust Synthetic Control](https://mlsanity.com/paper/1711.06940/) (2017)
- [How Case Based Reasoning Explained Neural Networks: An XAI Survey of Post-Hoc Explanation-by-Example in ANN-CBR Twins](https://mlsanity.com/paper/1905.07186/) (2019)
- [The Twin-System Approach as One Generic Solution for XAI: An Overview of ANN-CBR Twins for Explaining Deep Learning](https://mlsanity.com/paper/1905.08069/) (2019)
- [MARGIN: Uncovering Deep Neural Networks using Graph Signal Analysis](https://mlsanity.com/paper/1711.05407/) (2017)

## Built on by (most cited first, 30 of 30)

- [Concept Activation Regions: A Generalized Framework For Concept-Based Explanations](https://mlsanity.com/paper/2209.11222/) (2022)
- [Visual correspondence-based explanations improve AI robustness and human-AI team accuracy](https://mlsanity.com/paper/2208.00780/) (2022)
- [TANGOS: Regularizing Tabular Neural Networks through Gradient Orthogonalization and Specialization](https://mlsanity.com/paper/2303.05506/) (2023)
- [On the Challenges and Opportunities in Generative AI](https://mlsanity.com/paper/2403.00025/) (2024)
- [Benchmarking Heterogeneous Treatment Effect Models through the Lens of Interpretability](https://mlsanity.com/paper/2206.08363/) (2022)
- [Label-Free Explainability for Unsupervised Models](https://mlsanity.com/paper/2203.01928/) (2022)
- [Evaluating the Robustness of Interpretability Methods through Explanation Invariance and Equivariance](https://mlsanity.com/paper/2304.06715/) (2023)
- [Interpretable Machine Learning in Physics: A Review](https://mlsanity.com/paper/2503.23616/) (2025)
- [Spuriosity Rankings: Sorting Data to Measure and Mitigate Biases](https://mlsanity.com/paper/2212.02648/) (2022)
- [AutoPrognosis 2.0: Democratizing Diagnostic and Prognostic Modeling in Healthcare with Automated Machine Learning](https://mlsanity.com/paper/2210.12090/) (2022)
- [Accountability in Offline Reinforcement Learning: Explaining Decisions with a Corpus of Examples](https://mlsanity.com/paper/2310.07747/) (2023)
- [NAISR: A 3D Neural Additive Model for Interpretable Shape Representation](https://mlsanity.com/paper/2303.09234/) (2023)
- [What is Flagged in Uncertainty Quantification? Latent Density Models for Uncertainty Categorization](https://mlsanity.com/paper/2207.05161/) (2022)
- [Counterfactual Explanations via Locally-guided Sequential Algorithmic Recourse](https://mlsanity.com/paper/2309.04211/) (2023)
- [Automated Ensemble Multimodal Machine Learning for Healthcare](https://mlsanity.com/paper/2407.18227/) (2024)
- [Advancing Post Hoc Case Based Explanation with Feature Highlighting](https://mlsanity.com/paper/2311.03246/) (2023)
- [Interpreting Differentiable Latent States for Healthcare Time-series Data](https://mlsanity.com/paper/2311.17560/) (2023)
- [Enhancing Interpretability Through Loss-Defined Classification Objective in Structured Latent Spaces](https://mlsanity.com/paper/2412.08515/) (2024)
- [Optimising Human-AI Collaboration by Learning Convincing Explanations](https://mlsanity.com/paper/2311.07426/) (2023)
- [Data Science Principles for Interpretable and Explainable AI](https://mlsanity.com/paper/2405.10552/) (2024)
- [Interpretable Multimodal Learning for Tumor Protein-Metal Binding: Progress, Challenges, and Perspectives](https://mlsanity.com/paper/2504.03847/) (2025)
- [Explanations Go Linear: Post-hoc Explainability for Tabular Data with Interpretable Meta-Encoding](https://mlsanity.com/paper/2504.20667/) (2025)
- [Explainable reinforcement learning from human feedback to improve alignment](https://mlsanity.com/paper/2512.13837/) (2025)
- [Dual feature-based and example-based explanation methods](https://mlsanity.com/paper/2401.16294/) (2024)
- [Retrieve to Explain: Evidence-driven Predictions for Explainable Drug Target Identification](https://mlsanity.com/paper/2402.04068/) (2024)
- [Enabling Regional Explainability by Automatic and Model-agnostic Rule Extraction](https://mlsanity.com/paper/2406.17885/) (2024)
- [Learning local discrete features in explainable-by-design convolutional neural networks](https://mlsanity.com/paper/2411.00139/) (2024)
- [LucidAtlas: Learning Uncertainty-Aware, Covariate-Disentangled, Individualized Atlas Representations](https://mlsanity.com/paper/2502.08445/) (2025)
- [Towards Visually Explaining Statistical Tests with Applications in Biomedical Imaging](https://mlsanity.com/paper/2601.13899/) (2026)
- [XSPLAIN: XAI-enabling Splat-based Prototype Learning for Attribute-aware INterpretability](https://mlsanity.com/paper/2602.10239/) (2026)

## Abstract (arXiv)

Modern machine learning models are complicated. Most of them rely on convoluted latent representations of their input to issue a prediction. To achieve greater transparency than a black-box that connects inputs to predictions, it is necessary to gain a deeper understanding of these latent representations. To that aim, we propose SimplEx: a user-centred method that provides example-based explanations with reference to a freely selected set of examples, called the corpus. SimplEx uses the corpus to improve the user's understanding of the latent space with post-hoc explanations answering two questions: (1) Which corpus examples explain the prediction issued for a given test example? (2) What features of these corpus examples are relevant for the model to relate them to the test example? SimplEx provides an answer by reconstructing the test latent representation as a mixture of corpus latent representations. Further, we propose a novel approach, the Integrated Jacobian, that allows SimplEx to make explicit the contribution of each corpus feature in the mixture. Through experiments on tasks ranging from mortality prediction to image classification, we demonstrate that these decompositions are robust and accurate. With illustrative use cases in medicine, we show that SimplEx empowers the user by highlighting relevant patterns in the corpus that explain model representations. Moreover, we demonstrate how the freedom in choosing the corpus allows the user to have personalized explanations in terms of examples that are meaningful for them.