# Statistically Significant Concept-based Explanation of Image Classifiers via Model Knockoffs

arXiv: https://arxiv.org/abs/2305.18362 · HTML: https://mlsanity.com/paper/2305.18362/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 24 references that are in the map.
- First posted on arXiv 27 May 2023.

## Builds on (most cited first, 18 of 18)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Concept Whitening for Interpretable Image Recognition](https://mlsanity.com/paper/2002.01650/) (2020)
- [Explaining Classifiers with Causal Concept Effect (CaCE)](https://mlsanity.com/paper/1907.07165/) (2019)
- [Panning for Gold: Model-X Knockoffs for High-dimensional Controlled Variable Selection](https://mlsanity.com/paper/1610.02351/) (2016)
- [Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations](https://mlsanity.com/paper/2011.12854/) (2020)
- [Guided Variational Autoencoder for Disentanglement Learning](https://mlsanity.com/paper/2004.01255/) (2020)
- [Now You See Me (CME): Concept-based Model Extraction](https://mlsanity.com/paper/2010.13233/) (2020)
- [A Framework for Learning Ante-hoc Explainable Models via Concepts](https://mlsanity.com/paper/2108.11761/) (2021)
- [Deep Knockoffs](https://mlsanity.com/paper/1811.06687/) (2018)
- [Generative causal explanations of black-box classifiers](https://mlsanity.com/paper/2006.13913/) (2020)
- [Debiasing Concept-based Explanations with Causal Analysis](https://mlsanity.com/paper/2007.11500/) (2020)
- [Debiased-CAM to mitigate image perturbations with faithful visual explanations of machine learning](https://mlsanity.com/paper/2012.05567/) (2020)
- [Unsupervised Causal Binary Concepts Discovery with VAE for Black-box Model Explanation](https://mlsanity.com/paper/2109.04518/) (2021)

## Built on by (most cited first, 4 of 4)

- [Incorporating Expert Rules into Neural Networks in the Framework of Concept-Based Learning](https://mlsanity.com/paper/2402.14726/) (2024)
- [FI-CBL: A Probabilistic Method for Concept-Based Learning with Expert Rules](https://mlsanity.com/paper/2406.19897/) (2024)
- [Survival Concept-Based Learning Models](https://mlsanity.com/paper/2502.05950/) (2025)
- [Knockoffs-based False Discovery Rate Control and Simplification for Deep Neural Networks](https://mlsanity.com/paper/2606.04404/) (2026)

## Abstract (arXiv)

A concept-based classifier can explain the decision process of a deep learning model by human-understandable concepts in image classification problems. However, sometimes concept-based explanations may cause false positives, which misregards unrelated concepts as important for the prediction task. Our goal is to find the statistically significant concept for classification to prevent misinterpretation. In this study, we propose a method using a deep learning model to learn the image concept and then using the Knockoff samples to select the important concepts for prediction by controlling the False Discovery Rate (FDR) under a certain value. We evaluate the proposed method in our synthetic and real data experiments. Also, it shows that our method can control the FDR properly while selecting highly interpretable concepts to improve the trustworthiness of the model.