# Measuring Semantic Information Production in Generative Diffusion Models

arXiv: https://arxiv.org/abs/2506.10433 · HTML: https://mlsanity.com/paper/2506.10433/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 13 of its 15 references that are in the map.
- First posted on arXiv 12 Jun 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 13 of 13)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Conditional Image Generation with PixelCNN Decoders](https://mlsanity.com/paper/1606.05328/) (2016)
- [PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications](https://mlsanity.com/paper/1701.05517/) (2017)
- [Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models](https://mlsanity.com/paper/2404.07724/) (2024)
- [Dynamical Regimes of Diffusion Models](https://mlsanity.com/paper/2402.18491/) (2024)
- [A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Data](https://mlsanity.com/paper/2402.16991/) (2024)
- [Spontaneous Symmetry Breaking in Generative Diffusion Models](https://mlsanity.com/paper/2305.19693/) (2023)
- [Critical windows: non-asymptotic theory for feature emergence in diffusion models](https://mlsanity.com/paper/2403.01633/) (2024)
- [The statistical thermodynamics of generative diffusion models: Phase transitions, symmetry breaking and critical instability](https://mlsanity.com/paper/2310.17467/) (2023)
- [Dynamic Negative Guidance of Diffusion Models](https://mlsanity.com/paper/2410.14398/) (2024)

## Built on by (most cited first, 6 of 6)

- [Concurrence of Symmetry Breaking and Nonlocality Phase Transitions in Diffusion Models](https://mlsanity.com/paper/2605.04830/) (2026)
- [Latent Structure Emergence in Diffusion Models via Confidence-Based Filtering](https://mlsanity.com/paper/2602.06155/) (2026)
- [Spectral Guidance for Flexible and Efficient Control of Diffusion Models](https://mlsanity.com/paper/2605.28900/) (2026)
- [Information-Theoretic Classifier-Free Guidance with Adaptive Schedule Optimization](https://mlsanity.com/paper/2606.24025/) (2026)
- [Breakdown of Local Denoising as Semantic Speciation](https://mlsanity.com/paper/2609.38176/) (2026)
- [When Does a Diffusion Model Decide What to Draw ?](https://mlsanity.com/paper/2610.05645/) (2026)

## Abstract (arXiv)

It is well known that semantic and structural features of the generated images emerge at different times during the reverse dynamics of diffusion, a phenomenon that has been connected to physical phase transitions in magnets and other materials. In this paper, we introduce a general information-theoretic approach to measure when these class-semantic "decisions" are made during the generative process. By using an online formula for the optimal Bayesian classifier, we estimate the conditional entropy of the class label given the noisy state. We then determine the time intervals corresponding to the highest information transfer between noisy states and class labels using the time derivative of the conditional entropy. We demonstrate our method on one-dimensional Gaussian mixture models and on DDPM models trained on the CIFAR10 dataset. As expected, we find that the semantic information transfer is highest in the intermediate stages of diffusion while vanishing during the final stages. However, we found sizable differences between the entropy rate profiles of different classes, suggesting that different "semantic decisions" are located at different intermediate times.