mlsanityData to 7 Oct 2026

Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models

arXiv 2304.12526 ↗First posted 25 Apr 2023cs.CV cs.LGcited by 131

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models are powerful, but they require a lot of time and data to train. We propose Patch Diffusion, a generic patch-wise training framework, to significantly reduce the training time costs while improving data efficiency, which thus helps democratize diffusion model training to broader users. At the core of our innovations is a new conditional score function at the patch level, where the patch location in the original image is included as additional coordinate channels, while the patch size is randomized and diversified throughout training to encode the cross-region dependency at multiple scales. Sampling with our method is as easy as in the original diffusion model. Through Patch Diffusion, we could achieve ≥ 2× faster training, while maintaining comparable or better generation quality. Patch Diffusion meanwhile improves the performance of diffusion models trained on relatively small datasets, e.g., as few as 5,000 images to train from scratch. We achieve outstanding FID scores in line with state-of-the-art benchmarks: 1.77 on CelebA-64×64, 1.93 on AFHQv2-Wild-64×64, and 2.72 on ImageNet-256×256. We share our code and pre-trained models at https://github.com/Zhendong-Wang/Patch-Diffusion.

Key facts. Cited by 131 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 53 of its 66 references that are in the map. First posted on arXiv 25 Apr 2023.

Builds on

53 of 66 references.

Closest ancestors · 20

Same direction, newest first

  1. 2022SinDDM: A Single Image Denoising Diffusion Model29 Nov
  2. 2022SinDiffusion: Learning a Diffusion Model from a Single Natural Image22 Nov
  3. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models2 Nov
  4. 2022Soft Diffusion: Score Matching for General Corruptions12 Sep
  5. 2022Classifier-Free Diffusion Guidance26 Jul
  6. 2022Estimating the Optimal Covariance with Imperfect Mean in Diffusion Probabilistic Models15 Jun
  7. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  8. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  9. 2022Pseudo Numerical Methods for Diffusion Models on Manifolds20 Feb
  10. 2022Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models17 Jan
  11. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  12. 2021Score-based Generative Modeling in Latent Space10 Jun
8 more
  1. 2021Soft Truncation: A Universal Training Technique of Score-based Diffusion Model for High Precision Score Estimation10 Jun
  2. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  3. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  4. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  6. 2020Denoising Diffusion Implicit Models6 Oct
  7. 2020Denoising Diffusion Probabilistic Models19 Jun
  8. 2020Improved Techniques for Training Score-Based Generative Models16 Jun

Other references · 33

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  6. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  7. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  8. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  9. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  10. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  11. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  12. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
21 more
  1. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  2. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  3. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
  4. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  5. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 778
  6. 2015Holistically-Nested Edge Detectioncited by 590
  7. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 543
  8. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 445
  9. 2020Differentiable Augmentation for Data-Efficient GAN Trainingcited by 361
  10. 2021Few-shot Image Generation via Cross-domain Correspondencecited by 206
  11. 2022Diffusion-GAN: Training GANs with Diffusioncited by 203
  12. 2022SINE: SINgle Image Editing with Text-to-Image Diffusion Modelscited by 159
  13. 2020Adversarial Generation of Continuous Imagescited by 135
  14. 2020Few-shot Image Generation with Elastic Weight Consolidationcited by 122
  15. 2021Regularizing Generative Adversarial Networks under Limited Datacited by 105
  16. 2022Any-resolution Training for High-resolution Image Synthesiscited by 72
  17. 2019COCO-GAN: Generation by Parts via Conditional Coordinatingcited by 67
  18. 2022SinFusion: Training Diffusion Models on a Single Image or Videocited by 58
  19. 2021Anycost GANs for Interactive Image Synthesis and Editingcited by 54
  20. 2021Data-Efficient GAN Training Beyond (Just) Augmentations: A Lottery Ticket Perspectivecited by 25
  21. 2022Memory Efficient Patch-based Training for INR-based GANscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 18; 2024: 45; 2025: 51; 2026: 17.

131 papers.

Sort:

By area:

  1. 2024Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generationcited by 211
  2. 2023Structural Pruning for Diffusion Modelscited by 193
  3. 2023Fast Training of Diffusion Models with Masked Transformerscited by 172
  4. 2023Diffusion Models for Image Restoration and Enhancement: A Comprehensive Surveycited by 108
  5. 2023In-Context Learning Unlocked for Diffusion Modelscited by 103
  6. 2023Improving Tuning-Free Real Image Editing with Proximal Guidancecited by 87
  7. 2025PixNerd: Pixel Neural Field Diffusioncited by 75
  8. 2024One-Step Diffusion Distillation through Score Implicit Matchingcited by 68
  9. 2024MindBridge: A Cross-Subject Brain Decoding Frameworkcited by 65
  10. 2024Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributionscited by 53
  11. 2024Adversarial Score identity Distillation: Rapidly Surpassing the Teacher in One Stepcited by 47
  12. 2023Patched Denoising Diffusion Models For High-Resolution Image Synthesiscited by 45
38 more
  1. 2024Towards a Mechanistic Explanation of Diffusion Model Generalizationcited by 42
  2. 2023Taming Mode Collapse in Score Distillation for Text-to-3D Generationcited by 35
  3. 2024A Closer Look at Time Steps is Worthy of Triple Speed-Up for Diffusion Model Trainingcited by 30
  4. 2025Towards Better Alignment: Training Diffusion Models with Reinforcement Learning Against Sparse Rewardscited by 29
  5. 2024Hierarchical Patch Diffusion Models for High-Resolution Video Generationcited by 24
  6. 2023Fast Diffusion Modelcited by 20
  7. 2023Context Diffusion: In-Context Aware Image Generationcited by 20
  8. 2024A Survey on Diffusion Models for Recommender Systemscited by 20
  9. 2023A Survey on Generative Modeling with Limited Data, Few Shots, and Zero Shotcited by 19
  10. 2024Learning Image Priors through Patch-based Diffusion Models for Solving Inverse Problemscited by 18
  11. 2023Beta Diffusioncited by 17
  12. 2024Pruning then Reweighting: Towards Data-Efficient Training of Diffusion Modelscited by 17
  13. 2024Language-Driven 6-DoF Grasp Detection Using Negative Prompt Guidancecited by 16
  14. 2024Epsilon-VAE: Denoising as Visual Decodingcited by 16
  15. 2024Machine learning emulation of precipitation from km-scale UK regional climate simulations using a diffusion modelcited by 15
  16. 2024Immiscible Diffusion: Accelerating Diffusion Training with Noise Assignmentcited by 14
  17. 2024REDUCIO! Generating 1K Video within 16 Seconds using Extremely Compressed Motion Latentscited by 14
  18. 2023Learning Stackable and Skippable LEGO Bricks for Efficient, Reconfigurable, and Variable-Resolution Diffusion Modelingcited by 13
  19. 2025Distinguished Quantized Guidance for Diffusion-based Sequence Recommendationcited by 13
  20. 2023Distilling ODE Solvers of Diffusion Models into Smaller Stepscited by 11
  21. 2024StainDiffuser: MultiTask Dual Diffusion Model for Virtual Stainingcited by 11
  22. 2026ELT: Elastic Looped Transformers for Visual Generationcited by 11
  23. 2024ZeroDiff: Solidified Visual-Semantic Correlation in Zero-Shot Learningcited by 10
  24. 2026Stable Velocity: A Variance Perspective on Flow Matchingcited by 10
  25. 2023Image Synthesis under Limited Data: A Survey and Taxonomycited by 9
  26. 2024Patch-Based Diffusion Models Beat Whole-Image Models for Mismatched Distribution Inverse Problemscited by 8
  27. 2024Structured Diffusion Models with Mixture of Gaussians as Prior Distributioncited by 8
  28. 2025Local Diffusion Models and Phases of Data Distributionscited by 8
  29. 2025CardiacFlow: 3D+t Four-Chamber Cardiac Shape Completion and Generation via Flow Matchingcited by 8
  30. 2026TC-Pad'e: Trajectory-Consistent Pad'e Approximation for Diffusion Accelerationcited by 8
  31. 2023Masked Diffusion Models Are Fast Distribution Learnerscited by 7
  32. 2024MedDiff-FM: A Diffusion-based Foundation Model for Versatile Medical Image Applicationscited by 7
  33. 2025Accelerating Diffusion Model Training under Minimal Budgets: A Condensation-Based Perspectivecited by 7
  34. 2025TinySR: Pruning Diffusion for Real-World Image Super-Resolutioncited by 7
  35. 2024GLA-Grad: A Griffin-Lim Extended Waveform Generation Diffusion Modelcited by 6
  36. 2024Relation Rectification in Diffusion Modelcited by 6
  37. 2025Efficient Flow Matching using Latent Variablescited by 6
  38. 2025Foresight: Adaptive Layer Reuse for Accelerated and High-Quality Text-to-Video Generationcited by 6

50 most cited of 131 All 131 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022Improving Diffusion Model Efficiency Through Patching
  2. 2026Efficient and Training-Free Single-Image Diffusion Models
  3. 2025APT: Improving Diffusion Models for High Resolution Image Generation with Adaptive Path Tracing
  4. 2023DiffFit: Unlocking Transferability of Large Diffusion Models via Simple Parameter-Efficient Fine-Tuning
  5. 2023Adaptive Training Meets Progressive Scaling: Elevating Efficiency in Diffusion Models
  6. 2024A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models
  7. 2023Analyzing and Improving the Training Dynamics of Diffusion Models
  8. 2023Reflected Diffusion Models