# Towards interpretable-by-design deep learning algorithms

arXiv: https://arxiv.org/abs/2311.11396 · HTML: https://mlsanity.com/paper/2311.11396/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 66 references that are in the map.
- First posted on arXiv 19 Nov 2023.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification](https://mlsanity.com/paper/1709.00029/) (2017)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Do Better ImageNet Models Transfer Better?](https://mlsanity.com/paper/1805.08974/) (2018)
- [S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learning](https://mlsanity.com/paper/2207.12819/) (2022)
- [Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictions](https://mlsanity.com/paper/1710.04806/) (2017)
- [Neural Prototype Trees for Interpretable Fine-grained Image Recognition](https://mlsanity.com/paper/2012.02046/) (2020)
- [Revisiting Weakly Supervised Pre-Training of Visual Perception Models](https://mlsanity.com/paper/2201.08371/) (2022)
- [Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention](https://mlsanity.com/paper/1703.10631/) (2017)
- [Best Subset Selection via a Modern Optimization Lens](https://mlsanity.com/paper/1507.03133/) (2015)
- [B-cos Networks: Alignment is All We Need for Interpretability](https://mlsanity.com/paper/2205.10268/) (2022)
- [The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classification](https://mlsanity.com/paper/1503.01161/) (2015)
- [Towards Explainable Deep Neural Networks (xDNN)](https://mlsanity.com/paper/1912.02523/) (2019)
- [Exploratory Not Explanatory: Counterfactual Analysis of Saliency Maps for Deep Reinforcement Learning](https://mlsanity.com/paper/1912.05743/) (2019)
- [Visual Recognition with Deep Nearest Centroids](https://mlsanity.com/paper/2209.07383/) (2022)
- [Concept-level Debugging of Part-Prototype Networks](https://mlsanity.com/paper/2205.15769/) (2022)
- [Online Continual Learning with Contrastive Vision Transformer](https://mlsanity.com/paper/2207.13516/) (2022)
- [Clustering-based Domain-Incremental Learning](https://mlsanity.com/paper/2309.12078/) (2023)
- [On the Safety of Interpretable Machine Learning: A Maximum Deviation Approach](https://mlsanity.com/paper/2211.01498/) (2022)
- [An Interpretable Deep Semantic Segmentation Method for Earth Observation](https://mlsanity.com/paper/2210.12820/) (2022)

## Built on by (most cited first, 3 of 3)

- [Detecting Multimedia Generated by Large AI Models: A Survey](https://mlsanity.com/paper/2402.00045/) (2024)
- [Restyling Unsupervised Concept Based Interpretable Networks with Generative Models](https://mlsanity.com/paper/2407.01331/) (2024)
- [Unsupervised Domain Adaptation within Deep Foundation Latent Spaces](https://mlsanity.com/paper/2402.14976/) (2024)

## Abstract (arXiv)

The proposed framework named IDEAL (Interpretable-by-design DEep learning ALgorithms) recasts the standard supervised classification problem into a function of similarity to a set of prototypes derived from the training data, while taking advantage of existing latent spaces of large neural networks forming so-called Foundation Models (FM). This addresses the issue of explainability (stage B) while retaining the benefits from the tremendous achievements offered by DL models (e.g., visual transformers, ViT) pre-trained on huge data sets such as IG-3.6B + ImageNet-1K or LVD-142M (stage A). We show that one can turn such DL models into conceptually simpler, explainable-through-prototypes ones. The key findings can be summarized as follows: (1) the proposed models are interpretable through prototypes, mitigating the issue of confounded interpretations, (2) the proposed IDEAL framework circumvents the issue of catastrophic forgetting allowing efficient class-incremental learning, and (3) the proposed IDEAL approach demonstrates that ViT architectures narrow the gap between finetuned and non-finetuned models allowing for transfer learning in a fraction of time without finetuning of the feature space on a target dataset with iterative supervised methods.