mlsanityData to 7 Oct 2026

On the Separability of Information in Diffusion Models

arXiv 2509.23937 ↗First posted 28 Sep 2025cs.LG cond-mat.stat-mech cs.AIcited by 2

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models transform noise into data by injecting information that was captured in their neural network during the training phase. In this paper, we ask: textitwhat is this information? We find that, in pixel-space diffusion models, (1) a large fraction of the total information in the neural network is committed to reconstructing small-scale perceptual details of the image, and (2) the correlations between images and their class labels are informed by the semantic content of the images, and are largely agnostic to the low-level details. We argue that these properties are intrinsically tied to the manifold structure of the data itself. Finally, we show that these facts explain the efficacy of classifier-free guidance: the guidance vector amplifies the mutual information between images and conditioning signals early in the generative process, influencing semantic structure, but tapers out as perceptual details are filled in.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 32 of its 65 references that are in the map. First posted on arXiv 28 Sep 2025.

Builds on

32 of 65 references.

Closest ancestors · 24

Same direction, newest first

  1. 2025Locality in Image Diffusion Models Emerges from Data Statistics11 Sep
  2. 2025The Information Dynamics of Generative Diffusion27 Aug
  3. 2025Entropic Time Schedulers for Generative Diffusion Models18 Apr
  4. 2024Classifier-Free Guidance is a Predictor-Corrector16 Aug
  5. 2024On the Trajectory Regularity of ODE-based Diffusion Sampling18 May
  6. 2024Critical windows: non-asymptotic theory for feature emergence in diffusion models3 Mar
  7. 2023Characteristic Guidance: Non-linear Correction for Diffusion Model at Large Guidance Scale11 Dec
  8. 2023Interpretable Diffusion via Information Decomposition12 Oct
  9. 2023Generalization in diffusion models arises from geometry-adaptive harmonic representations4 Oct
  10. 2023Denoising Diffusion Autoencoders are Unified Self-supervised Learners17 Mar
  11. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation1 Mar
  12. 2023Information-Theoretic Diffusion7 Feb
12 more
  1. 2023Minimizing Trajectory Curvature of ODE-based Generative Models27 Jan
  2. 2022Classifier-Free Diffusion Guidance26 Jul
  3. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  5. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representation30 Nov
  6. 2021Variational Diffusion Models1 Jul
  7. 2021Score-based Generative Modeling in Latent Space10 Jun
  8. 2021A Variational Perspective on Diffusion-Based Generative Models and Score Matching5 Jun
  9. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  10. 2021Maximum Likelihood Training of Score-Based Diffusion Models22 Jan
  11. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  12. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 8

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  2. 2016Deep Variational Information Bottleneckcited by 915
  3. 2023Your Diffusion Model is Secretly a Zero-Shot Classifiercited by 301
  4. 2023Text-to-Image Diffusion Models are Zero-Shot Classifierscited by 140
  5. 2020Interacting particle solutions of Fokker-Planck equations through gradient-log-density estimationcited by 87
  6. 2018Auto-Encoding Total Correlation Explanationcited by 17
  7. 2024DiffusionPID: Interpreting Diffusion via Partial Information Decompositioncited by 16
  8. 2024Information Theoretic Text-to-Image Alignmentcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026The Diffusion Encodercited by 0
  2. 2026Breakdown of Local Denoising as Semantic Speciationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024How Diffusion Models Learn to Factorize and Compose
  2. 2026An exact information theory of generalization phase transitions in Bayesian diffusion models
  3. 2026Feature Information Dynamics in Diffusion
  4. 2024Theoretical Insights for Diffusion Guidance: A Case Study for Gaussian Mixture Models
  5. 2024Neural Entropy
  6. 2024Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributions
  7. 2025Measuring Semantic Information Production in Generative Diffusion Models
  8. 2025Understanding Representation Dynamics of Diffusion Models via Low-Dimensional Modeling