# Towards Fixing Clever-Hans Predictors with Counterfactual Knowledge Distillation

arXiv: https://arxiv.org/abs/2310.01011 · HTML: https://mlsanity.com/paper/2310.01011/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation.
- Builds on 12 of its 20 references that are in the map.
- First posted on arXiv 2 Oct 2023.

## Placed in

- Knowledge distillation / [General knowledge distillation methods](https://mlsanity.com/distill/general-knowledge-distillation-methods/)

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Spectral Signatures in Backdoor Attacks](https://mlsanity.com/paper/1811.00636/) (2018)
- [Unmasking Clever Hans Predictors and Assessing What Machines Really Learn](https://mlsanity.com/paper/1902.10178/) (2019)
- [Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations](https://mlsanity.com/paper/1703.03717/) (2017)
- [Learning Not to Learn: Training Deep Neural Networks with Biased Data](https://mlsanity.com/paper/1812.10352/) (2018)
- [Interpretations are useful: penalizing explanations to align neural networks with prior knowledge](https://mlsanity.com/paper/1909.13584/) (2019)
- [Making deep neural networks right for the right scientific reasons by interacting with their explanations](https://mlsanity.com/paper/2001.05371/) (2020)
- [Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs](https://mlsanity.com/paper/1801.05453/) (2018)
- [Diffeomorphic Counterfactuals with Generative Models](https://mlsanity.com/paper/2206.05075/) (2022)

## Built on by (most cited first, 14 of 14)

- [Towards Desiderata-Driven Design of Visual Counterfactual Explainers](https://mlsanity.com/paper/2506.14698/) (2025)
- [Mitigating Clever Hans Strategies in Image Classifiers through Generating Counterexamples](https://mlsanity.com/paper/2510.17524/) (2025)
- [Diffusion Counterfactuals for Image Regressors](https://mlsanity.com/paper/2503.20595/) (2025)
- [Visual Disentangled Diffusion Autoencoders: Scalable Counterfactual Generation for Foundation Models](https://mlsanity.com/paper/2601.21851/) (2026)
- [Imbalanced Classification through the Lens of Spurious Correlations](https://mlsanity.com/paper/2510.27650/) (2025)
- [ParkDiffusion++: Ego Intention Conditioned Joint Multi-Agent Trajectory Prediction for Automated Parking using Diffusion Models](https://mlsanity.com/paper/2602.20923/) (2026)
- [SCE-LITE-HQ: Smooth visual counterfactual explanations with generative foundation models](https://mlsanity.com/paper/2603.17048/) (2026)
- [Beyond Attention Heatmaps: How to Get Better Explanations for Multiple Instance Learning Models in Histopathology](https://mlsanity.com/paper/2603.08328/) (2026)
- [PINNfluence: Interpreting PINNs through Influence Functions](https://mlsanity.com/paper/2409.08958/) (2024)
- [Investigating the Robustness of Subtask Distillation under Spurious Correlation](https://mlsanity.com/paper/2602.00852/) (2026)
- [Faithful, Sufficient and Understandable: Rethinking Graph Counterfactual Explanations via Discrete Diffusion Inversion](https://mlsanity.com/paper/2608.12083/) (2026)
- [Training on Plausible Counterfactuals Removes Spurious Correlations](https://mlsanity.com/paper/2505.16583/) (2025)
- [Density-Guided Robust Counterfactual Explanations on Tabular Data under Model Multiplicity](https://mlsanity.com/paper/2605.30901/) (2026)
- [Towards Fast and Disentangled Counterfactuals for Visual Foundation Models](https://mlsanity.com/paper/2610.00895/) (2026)

## Abstract (arXiv)

This paper introduces a novel technique called counterfactual knowledge distillation (CFKD) to detect and remove reliance on confounders in deep learning models with the help of human expert feedback. Confounders are spurious features that models tend to rely on, which can result in unexpected errors in regulated or safety-critical domains. The paper highlights the benefit of CFKD in such domains and shows some advantages of counterfactual explanations over other types of explanations. We propose an experiment scheme to quantitatively evaluate the success of CFKD and different teachers that can give feedback to the model. We also introduce a new metric that is better correlated with true test performance than validation accuracy. The paper demonstrates the effectiveness of CFKD on synthetically augmented datasets and on real-world histopathological datasets.