# Explaining generative diffusion models via visual analysis for interpretable decision-making process

arXiv: https://arxiv.org/abs/2402.10404 · HTML: https://mlsanity.com/paper/2402.10404/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 25 of its 44 references that are in the map.
- First posted on arXiv 16 Feb 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 25 of 25)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [Deep Visual-Semantic Alignments for Generating Image Descriptions](https://mlsanity.com/paper/1412.2306/) (2014)
- [Blended Diffusion for Text-driven Editing of Natural Images](https://mlsanity.com/paper/2111.14818/) (2021)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [Diffusion Models already have a Semantic Latent Space](https://mlsanity.com/paper/2210.10960/) (2022)
- [Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers](https://mlsanity.com/paper/2103.15679/) (2021)
- [Perception Prioritized Training of Diffusion Models](https://mlsanity.com/paper/2204.00227/) (2022)
- [Explaining NonLinear Classification Decisions with Deep Taylor Decomposition](https://mlsanity.com/paper/1512.02479/) (2015)
- [Stable Bias: Analyzing Societal Representations in Diffusion Models](https://mlsanity.com/paper/2303.11408/) (2023)
- [Black-box Explanation of Object Detectors via Saliency Maps](https://mlsanity.com/paper/2006.03204/) (2020)
- [Unleashing Transformers: Parallel Token Prediction with Discrete Absorbing Diffusion for Fast High-Resolution Image Generation from Vector-Quantized Codes](https://mlsanity.com/paper/2111.12701/) (2021)
- [High Fidelity Visualization of What Your Self-Supervised Representation Knows About](https://mlsanity.com/paper/2112.09164/) (2021)
- [Biases in Generative Art -- A Causal Look from the Lens of Art History](https://mlsanity.com/paper/2010.13266/) (2020)
- [Fault Diagnosis using eXplainable AI: a Transfer Learning-based Approach for Rotating Machinery exploiting Augmented Synthetic Data](https://mlsanity.com/paper/2210.02974/) (2022)
- [VS-CAM: Vertex Semantic Class Activation Mapping to Interpret Vision Graph Neural Network](https://mlsanity.com/paper/2209.09104/) (2022)

## Built on by (most cited first, 17 of 17)

- [FastCache: Fast Caching for Diffusion Transformer Through Learnable Linear Approximation](https://mlsanity.com/paper/2505.20353/) (2025)
- [Beta Sampling is All You Need: Efficient Image Generation Strategy for Diffusion Models using Stepwise Spectral Analysis](https://mlsanity.com/paper/2407.12173/) (2024)
- [Concept-Based Explanations in Computer Vision: Where Are We and Where Could We Go?](https://mlsanity.com/paper/2409.13456/) (2024)
- [Unveiling Structural Memorization: Structural Membership Inference Attack for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2407.13252/) (2024)
- [Precise Parameter Localization for Textual Generation in Diffusion Models](https://mlsanity.com/paper/2502.09935/) (2025)
- [EvolvED: Evolutionary Embeddings to Understand the Generation Process of Diffusion Models](https://mlsanity.com/paper/2406.17462/) (2024)
- [X-Edit: Detecting and Localizing Edits in Images Altered by Text-Guided Diffusion Models](https://mlsanity.com/paper/2505.11753/) (2025)
- [SingVisio: Visual Analytics of Diffusion Model for Singing Voice Conversion](https://mlsanity.com/paper/2402.12660/) (2024)
- [A Roadmap Towards Automated and Regulated Robotic Systems](https://mlsanity.com/paper/2403.14049/) (2024)
- [MANTA -- Model Adapter Native generations that's Affordable](https://mlsanity.com/paper/2409.14363/) (2024)
- [InJecteD: Analyzing Trajectories and Drift Dynamics in Denoising Diffusion Probabilistic Models for 2D Point Cloud Generation](https://mlsanity.com/paper/2509.12239/) (2025)
- [STARE: Step-wise Temporal Alignment and Red-teaming Engine for Multi-modal Toxicity Attack](https://mlsanity.com/paper/2605.00699/) (2026)
- [Attention Dynamics in Diffusion Models: A Visual Analytics Framework for Human-AI Collaboration](https://mlsanity.com/paper/2607.02563/) (2026)
- [Tree-of-Thoughts Reasoning for Text-to-Image In-Context Learning](https://mlsanity.com/paper/2607.07117/) (2026)
- [HiLRP: Toward One Trustworthy Explanation for Vision Transformer: Conservation-Valid Attribution via Attention Primitives](https://mlsanity.com/paper/2609.01282/) (2026)
- [Learning to Explain While Planning: Rule-Aligned Diffusion Planning for Autonomous Driving](https://mlsanity.com/paper/2609.39995/) (2026)
- [SaTPO: Text-to-Image Safety Alignment via Transition Preference Optimization](https://mlsanity.com/paper/2610.19563/) (2026)

## Abstract (arXiv)

Diffusion models have demonstrated remarkable performance in generation tasks. Nevertheless, explaining the diffusion process remains challenging due to it being a sequence of denoising noisy images that are difficult for experts to interpret. To address this issue, we propose the three research questions to interpret the diffusion process from the perspective of the visual concepts generated by the model and the region where the model attends in each time step. We devise tools for visualizing the diffusion process and answering the aforementioned research questions to render the diffusion process human-understandable. We show how the output is progressively generated in the diffusion process by explaining the level of denoising and highlighting relationships to foundational visual concepts at each time step through the results of experiments with various visual analyses using the tools. Throughout the training of the diffusion model, the model learns diverse visual concepts corresponding to each time-step, enabling the model to predict varying levels of visual concepts at different stages. We substantiate our tools using Area Under Cover (AUC) score, correlation quantification, and cross-attention mapping. Our findings provide insights into the diffusion process and pave the way for further research into explainable diffusion mechanisms.