# Discovering Concept Directions from Diffusion-based Counterfactuals via Latent Clustering

arXiv: https://arxiv.org/abs/2505.07073 · HTML: https://mlsanity.com/paper/2505.07073/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 40 references that are in the map.
- First posted on arXiv 11 May 2025.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [Diffusion Autoencoders: Toward a Meaningful and Decodable Representation](https://mlsanity.com/paper/2111.15640/) (2021)
- [CRAFT: Concept Recursive Activation FacTorization for Explainability](https://mlsanity.com/paper/2211.10154/) (2022)
- [Explaining in Style: Training a GAN to explain a classifier in StyleSpace](https://mlsanity.com/paper/2104.13369/) (2021)
- [Concept-based Explainable Artificial Intelligence: A Survey](https://mlsanity.com/paper/2312.12936/) (2023)
- [Diffusion Visual Counterfactual Explanations](https://mlsanity.com/paper/2210.11841/) (2022)
- [Diffusion Models for Counterfactual Explanations](https://mlsanity.com/paper/2203.15636/) (2022)
- [Adversarial Counterfactual Visual Explanations](https://mlsanity.com/paper/2303.09962/) (2023)
- [Explainable AI for Robot Failures: Generating Explanations that Improve User Assistance in Fault Recovery](https://mlsanity.com/paper/2101.01625/) (2021)
- [CheXplaining in Style: Counterfactual Explanations for Chest X-rays using StyleGAN](https://mlsanity.com/paper/2207.07553/) (2022)
- [Understanding the (Extra-)Ordinary: Validating Deep Model Decisions with Prototypical Concept-based Explanations](https://mlsanity.com/paper/2311.16681/) (2023)
- [Explaining AI-based Decision Support Systems using Concept Localization Maps](https://mlsanity.com/paper/2005.01399/) (2020)
- [Global Counterfactual Directions](https://mlsanity.com/paper/2404.12488/) (2024)
- [Generating Counterfactual Trajectories with Latent Diffusion Models for Concept Discovery](https://mlsanity.com/paper/2404.10356/) (2024)
- [CoLa-DCE -- Concept-guided Latent Diffusion Counterfactual Explanations](https://mlsanity.com/paper/2406.01649/) (2024)
- [DifCluE: Generating Counterfactual Explanations with Diffusion Autoencoders and modal clustering](https://mlsanity.com/paper/2502.11509/) (2025)

## Abstract (arXiv)

Concept-based explanations have emerged as an effective approach within Explainable Artificial Intelligence, enabling interpretable insights by aligning model decisions with human-understandable concepts. However, existing methods rely on computationally intensive procedures and struggle to efficiently capture complex, semantic concepts. This work introduces the Concept Directions via Latent Clustering (CDLC), which extracts global, class-specific concept directions by clustering latent difference vectors derived from factual and diffusion-generated counterfactual image pairs. CDLC reduces storage requirements by ~4.6% and accelerates concept discovery by ~5.3% compared to the baseline method, while requiring no GPU for clustering, thereby enabling efficient extraction of multidimensional semantic concepts across latent dimensions. This approach is validated on a real-world skin lesion dataset, demonstrating that the extracted concept directions align with clinically recognized dermoscopic features and, in some cases, reveal dataset-specific biases or unknown biomarkers. These results highlight that CDLC is interpretable, scalable, and applicable across high-stakes domains and diverse data modalities.