mlsanityData to 7 Oct 2026

Perception Prioritized Training of Diffusion Models

arXiv 2204.00227 ↗First posted 1 Apr 2022cs.CV cs.LGcited by 301

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models learn to restore noisy data, which is corrupted with different levels of noise, by optimizing the weighted sum of the corresponding loss terms, i.e., denoising score matching loss. In this paper, we show that restoring data corrupted with certain noise levels offers a proper pretext task for the model to learn rich visual concepts. We propose to prioritize such noise levels over other levels during training, by redesigning the weighting scheme of the objective function. We show that our simple redesign of the weighting scheme significantly improves the performance of diffusion models regardless of the datasets, architectures, and sampling strategies.

Key facts. Cited by 301 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 43 of its 54 references that are in the map. First posted on arXiv 1 Apr 2022.

Builds on

43 of 54 references.

Closest ancestors · 11

Same direction, newest first

  1. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models6 Aug
  2. 2021Variational Diffusion Models1 Jul
  3. 2021Score-based Generative Modeling in Latent Space10 Jun
  4. 2021Learning to Efficiently Sample from Diffusion Probabilistic Models7 Jun
  5. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  6. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  7. 2021Maximum Likelihood Training of Score-Based Diffusion Models22 Jan
  8. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed7 Jan
  9. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  10. 2020Denoising Diffusion Implicit Models6 Oct
  11. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 32

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  5. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  6. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  7. 2013Auto-Encoding Variational Bayescited by 3,980
  8. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  9. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  10. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  11. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  12. 2016Density estimation using Real NVPcited by 2,029
20 more
  1. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  2. 2018Group Normalizationcited by 1,514
  3. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  4. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  5. 2018Demystifying MMD GANscited by 1,268
  6. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  7. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  8. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
  9. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  10. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  11. 2019CNN-generated images are surprisingly easy to spot... for nowcited by 842
  12. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  13. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  14. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 415
  15. 2020Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Imagescited by 261
  16. 2020Artificial Fingerprinting for Generative Models: Rooting Deepfake Attribution in Training Datacited by 200
  17. 2020Adversarial Latent Autoencoderscited by 155
  18. 2020A U-Net Based Discriminator for Generative Adversarial Networkscited by 128
  19. 2020VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Modelscited by 103
  20. 2021On Aliased Resizing and Surprising Subtleties in GAN Evaluationcited by 51

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 18; 2023: 78; 2024: 73; 2025: 70; 2026: 62.

301 papers.

Sort:

By area:

  1. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  2. 2022Diffusion Models in Vision: A Surveycited by 1,008
  3. 2023Exploiting Diffusion Prior for Real-World Image Super-Resolutioncited by 486
  4. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 455
  5. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  6. 2022Diffusion Models already have a Semantic Latent Spacecited by 324
  7. 2024MACE: Mass Concept Erasure in Diffusion Modelscited by 298
  8. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentationcited by 293
  9. 2022Diffusion Models for Medical Image Analysis: A Comprehensive Surveycited by 269
  10. 2023A Variational Perspective on Solving Inverse Problems with Diffusion Modelscited by 217
  11. 2022Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Modelscited by 215
  12. 2023Generative Diffusion Prior for Unified Image Restoration and Enhancementcited by 198
38 more
  1. 2023TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Compositioncited by 182
  2. 2023Dense Text-to-Image Generation with Attention Modulationcited by 162
  3. 2024Consistency Policy: Accelerated Visuomotor Policies via Consistency Distillationcited by 159
  4. 2023Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolutioncited by 131
  5. 2024Lumina-Next: Making Lumina-T2X Stronger and Faster with Next-DiTcited by 130
  6. 2022GENIE: Higher-Order Denoising Diffusion Solverscited by 127
  7. 2022Diffusion Probabilistic Model Made Slimcited by 121
  8. 2023Understanding the Latent Space of Diffusion Models through the Lens of Riemannian Geometrycited by 121
  9. 2022Towards the Detection of Diffusion Model Deepfakescited by 111
  10. 2022Latent Diffusion for Language Generationcited by 110
  11. 2023DiffiT: Diffusion Vision Transformers for Image Generationcited by 104
  12. 2024Principled Probabilistic Imaging using Diffusion Models as Plug-and-Play Priorscited by 93
  13. 2023DreamTime: An Improved Optimization Strategy for Diffusion-Guided 3D Generationcited by 81
  14. 2022A Morphology Focused Diffusion Probabilistic Model for Synthesis of Histopathology Imagescited by 74
  15. 2023DiffusionRig: Learning Personalized Priors for Facial Appearance Editingcited by 74
  16. 2023PFGM++: Unlocking the Potential of Physics-Inspired Generative Modelscited by 73
  17. 2023AutoDiffusion: Training-Free Optimization of Time Steps and Architectures for Automated Diffusion Model Accelerationcited by 70
  18. 2023Texture Generation on 3D Meshes with Point-UV Diffusioncited by 63
  19. 2024Diffusion Models, Image Super-Resolution And Everything: A Surveycited by 63
  20. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 59
  21. 2022Unsupervised Representation Learning from Pre-trained Diffusion Probabilistic Modelscited by 53
  22. 2022Efficient Diffusion Models for Vision: A Surveycited by 50
  23. 2023Wuerstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Modelscited by 50
  24. 2024Balancing Act: Distribution-Guided Debiasing in Diffusion Modelscited by 50
  25. 2024QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuningcited by 48
  26. 2023Analyzing Bias in Diffusion-based Face Generation Modelscited by 45
  27. 2023Stable Target Field for Reduced Variance Score Estimation in Diffusion Modelscited by 44
  28. 2023OMS-DPM: Optimizing the Model Schedule for Diffusion Probabilistic Modelscited by 44
  29. 2023CycleNet: Rethinking Cycle Consistency in Text-Guided Diffusion for Image Manipulationcited by 41
  30. 2024Critical windows: non-asymptotic theory for feature emergence in diffusion modelscited by 41
  31. 2024MetaEarth: A Generative Foundation Model for Global-Scale Remote Sensing Image Generationcited by 41
  32. 2024Improved Noise Schedule for Diffusion Trainingcited by 38
  33. 2023Training-free Content Injection using h-space in Diffusion Modelscited by 36
  34. 2023Multi-Architecture Multi-Expert Diffusion Modelscited by 34
  35. 2023Debias Coarsely, Sample Conditionally: Statistical Downscaling through Optimal Transport and Probabilistic Diffusion Modelscited by 33
  36. 2023Addressing Negative Transfer in Diffusion Modelscited by 33
  37. 2023Faster Diffusion: Rethinking the Role of the Encoder for Diffusion Model Inferencecited by 33
  38. 2022f-DM: A Multi-stage Diffusion Model via Progressive Signal Transformationcited by 32

50 most cited of 301 All 301 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Scaling Properties of Diffusion Models for Perceptual Tasks
  2. 2024What Matters When Repurposing Diffusion Models for General Dense Perception Tasks?
  3. 2025Breaking the Likelihood-Quality Trade-off in Diffusion Models by Merging Pretrained Experts
  4. 2024Diff-2-in-1: Bridging Generation and Dense Perception with Diffusion Models
  5. 2023Reflected Diffusion Models
  6. 2025Accelerating Diffusion Model Training under Minimal Budgets: A Condensation-Based Perspective
  7. 2025Demystifying Diffusion Objectives: Reweighted Losses are Better Variational Bounds
  8. 2024Diffusion Models in Low-Level Vision: A Survey