# Generating Counterfactual Trajectories with Latent Diffusion Models for Concept Discovery arXiv: https://arxiv.org/abs/2404.10356 ยท HTML: https://mlsanity.com/paper/2404.10356/ ## Key facts - Cited by 8 papers in the map (data through 14 Oct 2026). - Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. - Builds on 29 of its 43 references that are in the map. - First posted on arXiv 16 Apr 2024. ## Placed in - Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/) ## Builds on (most cited first, 29 of 29) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016) - ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016) - [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015) - [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017) - [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017) - [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020) - [Pseudo Numerical Methods for Diffusion Models on Manifolds](https://mlsanity.com/paper/2202.09778/) (2022) - [Interpretable Counterfactual Explanations Guided by Prototypes](https://mlsanity.com/paper/1907.02584/) (2019) - [CRAFT: Concept Recursive Activation FacTorization for Explainability](https://mlsanity.com/paper/2211.10154/) (2022) - [Multi-Objective Counterfactual Explanations](https://mlsanity.com/paper/2004.11165/) (2020) - [Explaining in Style: Training a GAN to explain a classifier in StyleSpace](https://mlsanity.com/paper/2104.13369/) (2021) - [Invertible Concept-based Explanations for CNN Models with Non-negative Concept Activation Vectors](https://mlsanity.com/paper/2006.15417/) (2020) - [Concept-based Explainable Artificial Intelligence: A Survey](https://mlsanity.com/paper/2312.12936/) (2023) - [Diffusion Visual Counterfactual Explanations](https://mlsanity.com/paper/2210.11841/) (2022) - [Diffusion Causal Models for Counterfactual Estimation](https://mlsanity.com/paper/2202.10166/) (2022) - [Demystifying the Draft EU Artificial Intelligence Act](https://mlsanity.com/paper/2107.03721/) (2021) - [Diffusion Models for Counterfactual Explanations](https://mlsanity.com/paper/2203.15636/) (2022) - [What is Healthy? Generative Counterfactual Diffusion for Lesion Localization](https://mlsanity.com/paper/2207.12268/) (2022) - [Multi-dimensional concept discovery (MCD): A unifying framework with completeness guarantees](https://mlsanity.com/paper/2301.11911/) (2023) - [Learning Bottleneck Concepts in Image Classification](https://mlsanity.com/paper/2304.10131/) (2023) - [Adversarial Counterfactual Visual Explanations](https://mlsanity.com/paper/2303.09962/) (2023) - [Loss Functions for Neural Networks for Image Processing](https://mlsanity.com/paper/1511.08861/) (2015) - [DISSECT: Disentangled Simultaneous Explanations via Concept Traversals](https://mlsanity.com/paper/2105.15164/) (2021) - [CheXplaining in Style: Counterfactual Explanations for Chest X-rays using StyleGAN](https://mlsanity.com/paper/2207.07553/) (2022) - [Explaining AI-based Decision Support Systems using Concept Localization Maps](https://mlsanity.com/paper/2005.01399/) (2020) - [XAI Handbook: Towards a Unified Framework for Explainable AI](https://mlsanity.com/paper/2105.06677/) (2021) - [Img2Tab: Automatic Class Relevant Concept Discovery from StyleGAN Features for Explainable Image Classification](https://mlsanity.com/paper/2301.06324/) (2023) ## Built on by (most cited first, 8 of 8) - [Concept-Based Explanations in Computer Vision: Where Are We and Where Could We Go?](https://mlsanity.com/paper/2409.13456/) (2024) - [Physics-Inspired Generative Models in Medical Imaging: A Review](https://mlsanity.com/paper/2407.10856/) (2024) - [Visual Disentangled Diffusion Autoencoders: Scalable Counterfactual Generation for Foundation Models](https://mlsanity.com/paper/2601.21851/) (2026) - [From Causal Discovery to Dynamic Causal Inference in Neural Time Series](https://mlsanity.com/paper/2603.20980/) (2026) - [Beyond Attribution: Unified Concept-Level Explanations](https://mlsanity.com/paper/2410.12439/) (2024) - [LD-ViCE: Latent Diffusion Model for Video Counterfactual Explanations](https://mlsanity.com/paper/2509.08422/) (2025) - [Discovering Concept Directions from Diffusion-based Counterfactuals via Latent Clustering](https://mlsanity.com/paper/2505.07073/) (2025) - [Towards Fast and Disentangled Counterfactuals for Visual Foundation Models](https://mlsanity.com/paper/2610.00895/) (2026) ## Abstract (arXiv) Trustworthiness is a major prerequisite for the safe application of opaque deep learning models in high-stakes domains like medicine. Understanding the decision-making process not only contributes to fostering trust but might also reveal previously unknown decision criteria of complex models that could advance the state of medical research. The discovery of decision-relevant concepts from black box models is a particularly challenging task. This study proposes Concept Discovery through Latent Diffusion-based Counterfactual Trajectories (CDCT), a novel three-step framework for concept discovery leveraging the superior image synthesis capabilities of diffusion models. In the first step, CDCT uses a Latent Diffusion Model (LDM) to generate a counterfactual trajectory dataset. This dataset is used to derive a disentangled representation of classification-relevant concepts using a Variational Autoencoder (VAE). Finally, a search algorithm is applied to identify relevant concepts in the disentangled latent space. The application of CDCT to a classifier trained on the largest public skin lesion dataset revealed not only the presence of several biases but also meaningful biomarkers. Moreover, the counterfactuals generated within CDCT show better FID scores than those produced by a previously established state-of-the-art method, while being 12 times more resource-efficient. Unsupervised concept discovery holds great potential for the application of trustworthy AI and the further development of human knowledge in various domains. CDCT represents a further step in this direction.