mlsanityData to 7 Oct 2026

Demystifying Diffusion Objectives: Reweighted Losses are Better Variational Bounds

arXiv 2511.19664 ↗First posted 24 Nov 2025cs.LG stat.MLcited by 7

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We derive a new theoretical interpretation of the reweighted losses that are widely used for training diffusion models. Our method is based on constructing a cascade of time-dependent variational lower bounds on the data log-likelihood, that provably improves upon the standard evidence lower bound and results in reduced data-model KL-divergences. Combining such bounds gives rise to reweighted objectives that can be applied to any generative diffusion model including both continuous Gaussian diffusion and masked (discrete) diffusion models. Then, we showcase this framework in masked diffusion and report significant improvements over previous training losses in pixel-space image modeling, approaching sample quality comparable to continuous diffusion models. Our results also provide a theoretical justification for the simple weighting scheme widely used in masked image models.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 35 of its 37 references that are in the map. First posted on arXiv 24 Nov 2025.

Builds on

35 of 37 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025Effective and Efficient Masked Image Generation Models10 Mar
  2. 2024FiT: Flexible Vision Transformer for Diffusion Model19 Feb
  3. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation1 Mar
  4. 2022Scalable Diffusion Models with Transformers19 Dec
  5. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  6. 2022A Continuous Time Framework for Discrete Denoising Models30 May
  7. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  8. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  9. 2021Variational Diffusion Models1 Jul
  10. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  11. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  12. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
1 more
  1. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 22

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2022Flow Matching for Generative Modelingcited by 5,426
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  6. 2022Video Diffusion Modelscited by 2,280
  7. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  8. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  9. 2025Large Language Diffusion Modelscited by 872
  10. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  11. 2024Autoregressive Image Generation without Vector Quantizationcited by 644
  12. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
10 more
  1. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  2. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  3. 2025Dream 7B: Diffusion Large Language Modelscited by 515
  4. 2022DiGress: Discrete Denoising diffusion for graph generationcited by 503
  5. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  6. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 371
  7. 2022Diffsound: Discrete Diffusion Model for Text-to-sound Generationcited by 256
  8. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Samplingcited by 204
  9. 2024Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Designcited by 72
  10. 2025Fractal Generative Modelscited by 44

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026Rethinking the Design Space of Reinforcement Learning for Diffusion Models: On the Importance of Likelihood Estimation Beyond Loss Designcited by 17
  2. 2026Stop Training for the Worst: Progressive Unmasking Accelerates Masked Diffusion Trainingcited by 8
  3. 2026Generalized Discrete Diffusion from Snapshotscited by 3
  4. 2026Tuning the Implicit Regularizer of Masked Diffusion Language Models: Enhancing Generalization via Insights from k-Paritycited by 1
  5. 2026What Does a Discrete Diffusion Model Learn?cited by 1
  6. 2026BlockGen: Flexible Blockwise Sequence Modeling with Hybrid Samplerscited by 0
  7. 2026dQwen3.5: Hybrid-Attention Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Unmasking Bias in Diffusion Model Training
  2. 2023DiffEnc: Variational Diffusion with a Learned Encoder
  3. 2023Masked Diffusion Models Are Fast Distribution Learners
  4. 2023A Variational Perspective on Solving Inverse Problems with Diffusion Models
  5. 2025Diffusion Models: A Mathematical Introduction
  6. 2023Reflected Diffusion Models
  7. 2025Diagnosing and Improving Diffusion Models by Estimating the Optimal Loss Value
  8. 2022Understanding Diffusion Models: A Unified Perspective