# Diffusion Model with Perceptual Loss

arXiv: https://arxiv.org/abs/2401.00110 · HTML: https://mlsanity.com/paper/2401.00110/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 46 of its 63 references that are in the map.
- First posted on arXiv 30 Dec 2023.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 46)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [MVDream: Multi-view Diffusion for 3D Generation](https://mlsanity.com/paper/2308.16512/) (2023)

## Built on by (most cited first, 21 of 21)

- [SDXL-Lightning: Progressive Adversarial Diffusion Distillation](https://mlsanity.com/paper/2402.13929/) (2024)
- [Diffusion Adversarial Post-Training for One-Step Video Generation](https://mlsanity.com/paper/2501.08316/) (2025)
- [Diffusion Models and Representation Learning: A Survey](https://mlsanity.com/paper/2407.00783/) (2024)
- [AnimateDiff-Lightning: Cross-Model Diffusion Distillation](https://mlsanity.com/paper/2403.12706/) (2024)
- [Boosting Latent Diffusion with Perceptual Objectives](https://mlsanity.com/paper/2411.04873/) (2024)
- [Towards Dense and Accurate Radar Perception Via Efficient Cross-Modal Diffusion Model](https://mlsanity.com/paper/2403.08460/) (2024)
- [Visual Generation Without Guidance](https://mlsanity.com/paper/2501.15420/) (2025)
- [PixelGen: Improving Pixel Diffusion with Perceptual Supervision](https://mlsanity.com/paper/2602.02493/) (2026)
- [Adversarial Flow Models](https://mlsanity.com/paper/2511.22475/) (2025)
- [Aligning Diffusion Models with Noise-Conditioned Perception](https://mlsanity.com/paper/2406.17636/) (2024)
- [Information Theoretic Text-to-Image Alignment](https://mlsanity.com/paper/2405.20759/) (2024)
- [Continuous Adversarial Flow Models](https://mlsanity.com/paper/2604.11521/) (2026)
- [Preconditioned Flow Matching](https://mlsanity.com/paper/2603.02337/) (2026)
- [Image Synthesis with Class-Aware Semantic Diffusion Models for Surgical Scene Segmentation](https://mlsanity.com/paper/2410.23962/) (2024)
- [Studying Classifier(-Free) Guidance From a Classifier-Centric Perspective](https://mlsanity.com/paper/2503.10638/) (2025)
- [Flow Straight to Reality: Perceptually Consistent Flow Matching for Efficient Image Restoration](https://mlsanity.com/paper/2608.10544/) (2026)
- [MANTA -- Model Adapter Native generations that's Affordable](https://mlsanity.com/paper/2409.14363/) (2024)
- [D-CNN and VQ-VAE Autoencoders for Compression and Denoising of Industrial X-ray Computed Tomography Images](https://mlsanity.com/paper/2507.07704/) (2025)
- [Align Your Tangent: Training Better Consistency Models via Manifold-Aligned Tangents](https://mlsanity.com/paper/2510.00658/) (2025)
- [Synthetic FMCW Radar Range Azimuth Maps Augmentation with Generative Diffusion Model](https://mlsanity.com/paper/2601.06228/) (2026)
- [CGGS: Consistency-Augmented Geometric Gaussian Splatting for Ego-Centric 3D Scene Generation](https://mlsanity.com/paper/2607.03819/) (2026)

## Abstract (arXiv)

Diffusion models without guidance generate very unrealistic samples. Guidance is used ubiquitously, and previous research has attributed its effect to low-temperature sampling that improves quality by trading off diversity. However, this perspective is incomplete. Our research shows that the choice of the loss objective is the underlying reason raw diffusion models fail to generate desirable samples. In this paper, (1) our analysis shows that the loss objective plays an important role in shaping the learned distribution and the MSE loss derived from theories holds assumptions that misalign with data in practice; (2) we explain the effectiveness of guidance methods from a new perspective of perceptual supervision; (3) we validate our hypothesis by training a diffusion model with a novel self-perceptual loss objective and obtaining much more realistic samples without the need for guidance. We hope our work paves the way for future explorations of the diffusion loss objective.