# Counterfactual Explanations for Medical Image Classification and Regression using Diffusion Autoencoder

arXiv: https://arxiv.org/abs/2408.01571 · HTML: https://mlsanity.com/paper/2408.01571/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 58 references that are in the map.
- First posted on arXiv 2 Aug 2024.

## Builds on (most cited first, 30 of 39)

- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classification](https://mlsanity.com/paper/2110.14795/) (2021)
- [Designing an Encoder for StyleGAN Image Manipulation](https://mlsanity.com/paper/2102.02766/) (2021)
- [Diffusion Autoencoders: Toward a Meaningful and Decodable Representation](https://mlsanity.com/paper/2111.15640/) (2021)
- [CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT](https://mlsanity.com/paper/2004.09167/) (2020)
- [Diffusion Models already have a Semantic Latent Space](https://mlsanity.com/paper/2210.10960/) (2022)
- [Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review](https://mlsanity.com/paper/2010.10596/) (2020)
- [Synthetic Data Augmentation using GAN for Improved Liver Lesion Classification](https://mlsanity.com/paper/1801.02385/) (2018)
- [VerSe: A Vertebrae Labelling and Segmentation Benchmark for Multi-detector CT Images](https://mlsanity.com/paper/2001.09193/) (2020)
- [Medical Image Synthesis with Context-Aware Generative Adversarial Networks](https://mlsanity.com/paper/1612.05362/) (2016)
- [Generative Hierarchical Features from Synthesizing Images](https://mlsanity.com/paper/2007.10379/) (2020)
- [What is Healthy? Generative Counterfactual Diffusion for Lesion Localization](https://mlsanity.com/paper/2207.12268/) (2022)
- [Using StyleGAN for Visual Interpretability of Deep Learning Models on Medical Images](https://mlsanity.com/paper/2101.07563/) (2021)
- [Lumbar spine segmentation in MR images: a dataset and a public benchmark](https://mlsanity.com/paper/2306.12217/) (2023)
- [Diffusion Models for Counterfactual Generation and Anomaly Detection in Brain Images](https://mlsanity.com/paper/2308.02062/) (2023)
- [A Computed Tomography Vertebral Segmentation Dataset with Anatomical Variations and Multi-Vendor Scanner Data](https://mlsanity.com/paper/2103.06360/) (2021)
- [CheXplaining in Style: Counterfactual Explanations for Chest X-rays using StyleGAN](https://mlsanity.com/paper/2207.07553/) (2022)
- [LARGE: Latent-Based Regression through GAN Semantics](https://mlsanity.com/paper/2107.11186/) (2021)
- [DreaMR: Diffusion-driven Counterfactual Explanation for Functional MRI](https://mlsanity.com/paper/2307.09547/) (2023)
- [Diffusion-based Iterative Counterfactual Explanations for Fetal Ultrasound Image Quality Assessment](https://mlsanity.com/paper/2403.08700/) (2024)
- [Deep Learning to Quantify Pulmonary Edema in Chest Radiographs](https://mlsanity.com/paper/2008.05975/) (2020)
- [medXGAN: Visual Explanations for Medical Classifiers through a Generative Latent Space](https://mlsanity.com/paper/2204.05376/) (2022)
- [Counterfactual Image Synthesis for Discovery of Personalized Predictive Image Markers](https://mlsanity.com/paper/2208.02311/) (2022)
- [Keypoints Localization for Joint Vertebra Detection and Fracture Severity Quantification](https://mlsanity.com/paper/2005.11960/) (2020)
- [Grading Loss: A Fracture Grade-based Metric Loss for Vertebral Fracture Detection](https://mlsanity.com/paper/2008.07831/) (2020)

## Built on by (most cited first, 10 of 10)

- [Diffusion Counterfactual Generation with Semantic Abduction](https://mlsanity.com/paper/2506.07883/) (2025)
- [Diffusion Counterfactuals for Image Regressors](https://mlsanity.com/paper/2503.20595/) (2025)
- [Evaluation of 3D Counterfactual Brain MRI Generation](https://mlsanity.com/paper/2508.02880/) (2025)
- [Diffusion-based Counterfactual Augmentation: Towards Robust and Interpretable Knee Osteoarthritis Grading](https://mlsanity.com/paper/2506.15748/) (2025)
- [On the notion of missingness for path attribution explainability methods in medical settings: Guiding the selection of medically meaningful baselines](https://mlsanity.com/paper/2508.14482/) (2025)
- [Hide-and-Seek Attribution: Weakly Supervised Segmentation of Vertebral Metastases in CT](https://mlsanity.com/paper/2512.06849/) (2025)
- [UbiQVision: Quantifying Uncertainty in XAI for Image Recognition](https://mlsanity.com/paper/2512.20288/) (2025)
- [EvoMorph: Counterfactual Explanations for Continuous Time-Series Extrinsic Regression Applied to Photoplethysmography](https://mlsanity.com/paper/2601.10356/) (2026)
- [Counterfactual Explainability Framework With CycleGAN And Counterfactual-Classifier Alignnment Score for Retinal Disease Classification](https://mlsanity.com/paper/2607.21068/) (2026)
- [Generating Medical Image Counterfactuals using Causal Explanations](https://mlsanity.com/paper/2609.02697/) (2026)

## Abstract (arXiv)

Counterfactual explanations (CEs) aim to enhance the interpretability of machine learning models by illustrating how alterations in input features would affect the resulting predictions. Common CE approaches require an additional model and are typically constrained to binary counterfactuals. In contrast, we propose a novel method that operates directly on the latent space of a generative model, specifically a Diffusion Autoencoder (DAE). This approach offers inherent interpretability by enabling the generation of CEs and the continuous visualization of the model's internal representation across decision boundaries. Our method leverages the DAE's ability to encode images into a semantically rich latent space in an unsupervised manner, eliminating the need for labeled data or separate feature extraction models. We show that these latent representations are helpful for medical condition classification and the ordinal regression of severity pathologies, such as vertebral compression fractures (VCF) and diabetic retinopathy (DR). Beyond binary CEs, our method supports the visualization of ordinal CEs using a linear model, providing deeper insights into the model's decision-making process and enhancing interpretability. Experiments across various medical imaging datasets demonstrate the method's advantages in interpretability and versatility. The linear manifold of the DAE's latent space allows for meaningful interpolation and manipulation, making it a powerful tool for exploring medical image properties. Our code is available at https://doi.org/10.5281/zenodo.13859266.