mlsanityData to 7 Oct 2026

Breakdown of Local Denoising as Semantic Speciation

arXiv 2609.38176 ↗First posted 29 Sep 2026cs.LG cond-mat.dis-nn cond-mat.stat-mechcited by 0

Not on the map. Why?

Abstract (arXiv)

The dynamics of generative models exhibit two apparently distinct temporal windows: a speciation window, in which a sample commits to a semantic class, and a nonlocality window, in which local context windows become insufficient for generation. Motivated by evidence of their near-concurrence in a variety of frontier models, we investigate their relationship through the spatial distribution of semantic information. Under a "common cause" hypothesis, we prove that the nonlocality window must lie in the speciation window. This hypothesis postulates that semantic labels explain a fraction of the correlations between distant tokens, a condition that is natural for many real datasets. We further give conditions under which both windows shrink to a single limiting time as system size grows, defining a "phase transition", and verify this behavior analytically in Gaussian mixtures. Together, these results identify conditions under which semantic information explains the concurrence of speciation and nonlocality, connecting two complementary perspectives on the emergence of semantic structure in generative modeling.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 43 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

32 of 43 references.

References · 32

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2022Scalable Diffusion Models with Transformerscited by 5,931
  7. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  8. 2021Convex optimizationcited by 2,479
  9. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  10. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
  11. 2019On Variational Bounds of Mutual Informationcited by 550
  12. 2022Perception Prioritized Training of Diffusion Modelscited by 301
20 more
  1. 2024Dynamical Regimes of Diffusion Modelscited by 133
  2. 2024An analytic theory of creativity in convolutional diffusion modelscited by 103
  3. 2024A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Datacited by 88
  4. 2023Spontaneous Symmetry Breaking in Generative Diffusion Modelscited by 74
  5. 2024Towards a Mechanistic Explanation of Diffusion Model Generalizationcited by 42
  6. 2024Critical windows: non-asymptotic theory for feature emergence in diffusion modelscited by 41
  7. 2025Memorization to Generalization: Emergence of Diffusion Models from Associative Memorycited by 35
  8. 2025Locality in Image Diffusion Models Emerges from Data Statisticscited by 25
  9. 2025Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusioncited by 17
  10. 2025Blink of an eye: a simple theory for feature localization in generative modelscited by 10
  11. 2025Local Diffusion Models and Phases of Data Distributionscited by 8
  12. 2021A General Derivative Identity for the Conditional Mean Estimator in Gaussian Noise and Some Applicationscited by 7
  13. 2026The Entropic Signature of Class Speciation in Diffusion Modelscited by 7
  14. 2025Measuring Semantic Information Production in Generative Diffusion Modelscited by 5
  15. 2026How Out-of-Equilibrium Phase Transitions can Seed Pattern Formation in Trained Diffusion Modelscited by 5
  16. 2025MMG: Mutual Information Estimation via the MMSE Gap in Diffusioncited by 3
  17. 2026An exact information theory of generalization phase transitions in Bayesian diffusion modelscited by 3
  18. 2025On the Separability of Information in Diffusion Modelscited by 2
  19. 2026Dissecting Failure Dynamics in Large Language Model Reasoningcited by 2
  20. 2026Concurrence of Symmetry Breaking and Nonlocality Phase Transitions in Diffusion Modelscited by 1

Built on by

Not cited yet.