mlsanityData to 14 Oct 2026

Towards Fixing Clever-Hans Predictors with Counterfactual Knowledge Distillation

arXiv 2310.01011 ↗First posted 2 Oct 2023cs.AIcited by 14

On the map: Knowledge distillation → General knowledge distillation methods

Abstract (arXiv)

This paper introduces a novel technique called counterfactual knowledge distillation (CFKD) to detect and remove reliance on confounders in deep learning models with the help of human expert feedback. Confounders are spurious features that models tend to rely on, which can result in unexpected errors in regulated or safety-critical domains. The paper highlights the benefit of CFKD in such domains and shows some advantages of counterfactual explanations over other types of explanations. We propose an experiment scheme to quantitatively evaluate the success of CFKD and different teachers that can give feedback to the model. We also introduce a new metric that is better correlated with true test performance than validation accuracy. The paper demonstrates the effectiveness of CFKD on synthetically augmented datasets and on real-world histopathological datasets.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation. Builds on 12 of its 20 references that are in the map. First posted on arXiv 2 Oct 2023.

Builds on

12 of 20 references.

References · 12

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  3. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,545
  4. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  5. 2018Spectral Signatures in Backdoor Attackscited by 484
  6. 2019Unmasking Clever Hans Predictors and Assessing What Machines Really Learncited by 365
  7. 2017Right for the Right Reasons: Training Differentiable Models by Constraining their Explanationscited by 294
  8. 2018Learning Not to Learn: Training Deep Neural Networks with Biased Datacited by 255
  9. 2019Interpretations are useful: penalizing explanations to align neural networks with prior knowledgecited by 145
  10. 2020Making deep neural networks right for the right scientific reasons by interacting with their explanationscited by 142
  11. 2018Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMscited by 67
  12. 2022Diffeomorphic Counterfactuals with Generative Modelscited by 29

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 8.

14 papers.

Sort:

By area:

  1. 2025Towards Desiderata-Driven Design of Visual Counterfactual Explainerscited by 7
  2. 2025Mitigating Clever Hans Strategies in Image Classifiers through Generating Counterexamplescited by 7
  3. 2025Diffusion Counterfactuals for Image Regressorscited by 5
  4. 2026Visual Disentangled Diffusion Autoencoders: Scalable Counterfactual Generation for Foundation Modelscited by 4
  5. 2025Imbalanced Classification through the Lens of Spurious Correlationscited by 3
  6. 2026ParkDiffusion++: Ego Intention Conditioned Joint Multi-Agent Trajectory Prediction for Automated Parking using Diffusion Modelscited by 3
  7. 2026SCE-LITE-HQ: Smooth visual counterfactual explanations with generative foundation modelscited by 3
  8. 2026Beyond Attention Heatmaps: How to Get Better Explanations for Multiple Instance Learning Models in Histopathologycited by 2
  9. 2024PINNfluence: Interpreting PINNs through Influence Functionscited by 1
  10. 2026Investigating the Robustness of Subtask Distillation under Spurious Correlationcited by 1
  11. 2026Faithful, Sufficient and Understandable: Rethinking Graph Counterfactual Explanations via Discrete Diffusion Inversioncited by 1
  12. 2025Training on Plausible Counterfactuals Removes Spurious Correlationscited by 0
2 more
  1. 2026Density-Guided Robust Counterfactual Explanations on Tabular Data under Model Multiplicitycited by 0
  2. 2026Towards Fast and Disentangled Counterfactuals for Visual Foundation Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Few-Shot Knowledge Distillation of LLMs With Counterfactual Explanations
  2. 2026Generating Medical Image Counterfactuals using Causal Explanations
  3. 2025LeapFactual: Reliable Visual Counterfactual Explanation Using Conditional Flow Matching
  4. 2025Executable Counterfactuals: Improving LLMs' Causal Reasoning Through Code
  5. 2025Factored Classifier-Free Guidance
  6. 2026On the Hidden Costs of Counterfactual Knowledge Training in LLM Unlearning
  7. 2025Enhancing XAI Narratives through Multi-Narrative Refinement and Knowledge Distillation
  8. 2025V-CECE: Visual Counterfactual Explanations via Conceptual Edits