mlsanityData to 14 Oct 2026

Patronus: Interpretable Diffusion Models with Prototypes

arXiv 2503.22782 ↗First posted 28 Mar 2025cs.CV cs.AI cs.LGcited by 0

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Uncovering the opacity of diffusion-based generative models is urgently needed, as their applications continue to expand while their underlying procedures largely remain a black box. With a critical question -- how can the diffusion generation process be interpreted and understood? -- we proposed Patronus, an interpretable diffusion model that incorporates a prototypical network to encode semantics in visual patches, revealing what visual patterns are modeled and where and when they emerge throughout denoising. This interpretability of Patronus provides deeper insights into the generative mechanism, enabling the detection of shortcut learning via unwanted correlations and the tracing of semantic emergence across timesteps. We evaluate Patronus on four natural image datasets and one medical imaging dataset, demonstrating both faithful interpretability and strong generative performance. With this work, we open new avenues for understanding and steering diffusion models through prototype-based interpretability. Our code is available at https://github.com/nina-weng/patronushttps://github.com/nina-weng/patronus.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 23 of its 26 references that are in the map. First posted on arXiv 28 Mar 2025.

Builds on

23 of 26 references.

Closest ancestors · 8

Same direction, newest first

  1. 2023FreeU: Free Lunch in Diffusion U-Net20 Sep
  2. 2023Understanding the Latent Space of Diffusion Models through the Lens of Riemannian Geometry24 Jul
  3. 2023DiffuseGAE: Controllable and High-fidelity Image Manipulation from Disentangled Representation12 Jul
  4. 2023InfoDiffusion: Representation Learning Using Information Maximizing Diffusion Models14 Jun
  5. 2023Discovering Interpretable Directions in the Semantic Latent Space of Diffusion Models20 Mar
  6. 2022Diffusion Models already have a Semantic Latent Space20 Oct
  7. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representation30 Nov
  8. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 15

  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  4. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,423
  5. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 906
  6. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 737
  7. 2017Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictionscited by 234
  8. 2023Stable Bias: Analyzing Societal Representations in Diffusion Modelscited by 226
  9. 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Modelscited by 175
  10. 2021Deformable ProtoPNet: An Interpretable Image Classifier Using Deformable Prototypescited by 120
  11. 2023On Provable Copyright Protection for Generative Modelscited by 98
  12. 2019Interpretable and Steerable Sequence Learning via Prototypescited by 72
3 more
  1. 2023Learning Support and Trivial Prototypes for Interpretable Image Classificationcited by 38
  2. 2022NashAE: Disentangling Representations through Adversarial Covariance Minimizationcited by 13
  3. 2021Multi-Stage Prototype Learning for Interpretable Time Series Classificationcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Emergence and Evolution of Interpretable Concepts in Diffusion Models
  2. 2024Explaining generative diffusion models via visual analysis for interpretable decision-making process
  3. 2024textitRevelio: Interpreting and leveraging semantic information in diffusion models
  4. 2024Decoding Diffusion: A Scalable Framework for Unsupervised Analysis of Latent Space Biases and Representations Using Natural Language Prompts
  5. 2022Let us Build Bridges: Understanding and Extending Diffusion Generative Models
  6. 2023NoiseCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions in Diffusion Models
  7. 2023Trade-offs in Fine-tuned Diffusion Models Between Accuracy and Interpretability
  8. 2024DiffusionPID: Interpreting Diffusion via Partial Information Decomposition