mlsanityData to 7 Oct 2026

Sparsely Supervised Diffusion

arXiv 2602.02699 ↗First posted 2 Feb 2026cs.LG cs.AIcited by 0

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models have shown remarkable success across a wide range of generative tasks. However, they often suffer from spatially inconsistent generation, arguably due to the inherent locality of their denoising mechanisms. This can yield samples that are locally plausible but globally inconsistent. To mitigate this issue, we propose sparsely supervised learning for diffusion models, a simple yet effective masking strategy that can be implemented with only a few lines of code. Interestingly, the experiments show that it is safe to mask up to 98% of pixels during diffusion model training. Our method delivers competitive FID scores across experiments and, most importantly, avoids training instability on small datasets. Moreover, the masking strategy reduces memorization and promotes the use of essential contextual information during generation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 44 of its 58 references that are in the map. First posted on arXiv 2 Feb 2026.

Builds on

44 of 58 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025Locality in Image Diffusion Models Emerges from Data Statistics11 Sep
  2. 2025Why Diffusion Models Don't Memorize: The Role of Implicit Dynamical Regularization in Training23 May
  3. 2025An Analytical Theory of Spectral Bias in the Learning Dynamics of Diffusion Models5 Mar
  4. 2024Towards a Mechanistic Explanation of Diffusion Model Generalization28 Nov
  5. 2024Rethinking the Spatial Inconsistency in Classifier-Free Diffusion Guidance8 Apr
  6. 2024Dynamical Regimes of Diffusion Models28 Feb
  7. 2023On Memorization in Diffusion Models4 Oct
  8. 2023Generalization in diffusion models arises from geometry-adaptive harmonic representations4 Oct
  9. 2023Understanding and Mitigating Copying in Diffusion Models31 May
  10. 2023MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizer25 Mar
  11. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  12. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
3 more
  1. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  3. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  8. 2022Flow Matching for Generative Modelingcited by 5,426
  9. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  10. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  11. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  12. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
17 more
  1. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  2. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
  3. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  4. 2022Building Normalizing Flows with Stochastic Interpolantscited by 964
  5. 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 897
  6. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  7. 2016Pixel Recurrent Neural Networkscited by 814
  8. 2024Autoregressive Image Generation without Vector Quantizationcited by 644
  9. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  10. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  11. 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 345
  12. 2024Discrete Flow Matchingcited by 301
  13. 2024Flow Matching Guide and Codecited by 291
  14. 2022Generative Modelling With Inverse Heat Dissipationcited by 164
  15. 2024An analytic theory of creativity in convolutional diffusion modelscited by 103
  16. 2023GIVT: Generative Infinite-Vocabulary Transformerscited by 80
  17. 2024Detecting Multimedia Generated by Large AI Models: A Surveycited by 73

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Masked Diffusion Models Are Fast Distribution Learners
  2. 2025Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking
  3. 2023Fast Training of Diffusion Models with Masked Transformers
  4. 2025Sparse-to-Sparse Training of Diffusion Models
  5. 2025Dimathtt[M]O: Distilling Masked Diffusion Models into One-step Generator
  6. 2023Diffusion Models as Masked Autoencoders
  7. 2025Sparse-LaViDa: Sparse Multimodal Discrete Diffusion Language Models
  8. 2024Learning on Less: Constraining Pre-trained Model Learning for Generalizable Diffusion-Generated Image Detection