mlsanityData to 7 Oct 2026

Diffusion Model with Perceptual Loss

arXiv 2401.00110 ↗First posted 30 Dec 2023cs.CV cs.AI cs.LGcited by 21

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models without guidance generate very unrealistic samples. Guidance is used ubiquitously, and previous research has attributed its effect to low-temperature sampling that improves quality by trading off diversity. However, this perspective is incomplete. Our research shows that the choice of the loss objective is the underlying reason raw diffusion models fail to generate desirable samples. In this paper, (1) our analysis shows that the loss objective plays an important role in shaping the learned distribution and the MSE loss derived from theories holds assumptions that misalign with data in practice; (2) we explain the effectiveness of guidance methods from a new perspective of perceptual supervision; (3) we validate our hypothesis by training a diffusion model with a novel self-perceptual loss objective and obtaining much more realistic samples without the need for guidance. We hope our work paves the way for future explorations of the diffusion loss objective.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 46 of its 63 references that are in the map. First posted on arXiv 30 Dec 2023.

Builds on

46 of 63 references.

Closest ancestors · 22

Same direction, newest first

  1. 2023Guided Diffusion from Self-Supervised Diffusion Features14 Dec
  2. 2023Smooth Diffusion: Crafting Smooth Latent Spaces in Diffusion Models7 Dec
  3. 2023Analyzing and Improving the Training Dynamics of Diffusion Models5 Dec
  4. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis30 Sep
  5. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  6. 2023Common Diffusion Noise Schedules and Sample Steps are Flawed15 May
  7. 2023Efficient Diffusion Training via Min-SNR Weighting Strategy16 Mar
  8. 2023Simple diffusion: End-to-end diffusion for high resolution images26 Jan
  9. 2022Scalable Diffusion Models with Transformers19 Dec
  10. 2022Refining Generative Process with Discriminator Guidance in Score-based Diffusion Models28 Nov
  11. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models2 Nov
  12. 2022Improving Sample Quality of Diffusion Models Using Self-Attention Guidance3 Oct
10 more
  1. 2022Classifier-Free Diffusion Guidance26 Jul
  2. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  3. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  4. 2022Perception Prioritized Training of Diffusion Models1 Apr
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  6. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  7. 2021Maximum Likelihood Training of Score-Based Diffusion Models22 Jan
  8. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  9. 2020Denoising Diffusion Implicit Models6 Oct
  10. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 24

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 2022Generative Adversarial Networkscited by 5,603
  8. 2022Flow Matching for Generative Modelingcited by 5,426
  9. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  10. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  11. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  12. 2016Improved Techniques for Training GANscited by 2,938
12 more
  1. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  2. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  3. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  4. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  5. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  6. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  7. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  8. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
  9. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  10. 2022MoVQ: Modulating Quantized Vectors for High-Fidelity Image Generationcited by 125
  11. 2023Wuerstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Modelscited by 50
  12. 2023MagicProp: Diffusion-based Video Editing via Motion-aware Appearance Propagationcited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 6; 2026: 6.

21 papers.

Sort:

By area:

  1. 2024SDXL-Lightning: Progressive Adversarial Diffusion Distillationcited by 258
  2. 2025Diffusion Adversarial Post-Training for One-Step Video Generationcited by 142
  3. 2024Diffusion Models and Representation Learning: A Surveycited by 53
  4. 2024AnimateDiff-Lightning: Cross-Model Diffusion Distillationcited by 47
  5. 2024Boosting Latent Diffusion with Perceptual Objectivescited by 19
  6. 2024Towards Dense and Accurate Radar Perception Via Efficient Cross-Modal Diffusion Modelcited by 17
  7. 2025Visual Generation Without Guidancecited by 17
  8. 2025Adversarial Flow Modelscited by 10
  9. 2026PixelGen: Improving Pixel Diffusion with Perceptual Supervisioncited by 10
  10. 2024Aligning Diffusion Models with Noise-Conditioned Perceptioncited by 9
  11. 2024Information Theoretic Text-to-Image Alignmentcited by 8
  12. 2026Continuous Adversarial Flow Modelscited by 4
9 more
  1. 2026Preconditioned Flow Matchingcited by 2
  2. 2024Image Synthesis with Class-Aware Semantic Diffusion Models for Surgical Scene Segmentationcited by 1
  3. 2025Studying Classifier(-Free) Guidance From a Classifier-Centric Perspectivecited by 1
  4. 2026Flow Straight to Reality: Perceptually Consistent Flow Matching for Efficient Image Restorationcited by 1
  5. 2024MANTA -- Model Adapter Native generations that's Affordablecited by 0
  6. 2025D-CNN and VQ-VAE Autoencoders for Compression and Denoising of Industrial X-ray Computed Tomography Imagescited by 0
  7. 2025Align Your Tangent: Training Better Consistency Models via Manifold-Aligned Tangentscited by 0
  8. 2026Synthetic FMCW Radar Range Azimuth Maps Augmentation with Generative Diffusion Modelcited by 0
  9. 2026CGGS: Consistency-Augmented Geometric Gaussian Splatting for Ego-Centric 3D Scene Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Theoretical Insights for Diffusion Guidance: A Case Study for Gaussian Mixture Models
  2. 2023Bridging the Gap: Addressing Discrepancies in Diffusion Model Training for Classifier-Free Guidance
  3. 2025Diagnosing and Improving Diffusion Models by Estimating the Optimal Loss Value
  4. 2024Diffusion Attribution Score: Evaluating Training Data Influence in Diffusion Models
  5. 2026Diffusion Models, Denoiser Architecture and Creativity
  6. 2025Loss Functions in Diffusion Models: A Comparative Study
  7. 2024Non-Normal Diffusion Models
  8. 2023Unmasking Bias in Diffusion Model Training