mlsanityData to 7 Oct 2026

DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models

arXiv 2402.19481 ↗First posted 29 Feb 2024cs.CVcited by 98

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models have achieved great success in synthesizing high-quality images. However, generating high-resolution images with diffusion models is still challenging due to the enormous computational costs, resulting in a prohibitive latency for interactive applications. In this paper, we propose DistriFusion to tackle this problem by leveraging parallelism across multiple GPUs. Our method splits the model input into multiple patches and assigns each patch to a GPU. However, naively implementing such an algorithm breaks the interaction between patches and loses fidelity, while incorporating such an interaction will incur tremendous communication overhead. To overcome this dilemma, we observe the high similarity between the input from adjacent diffusion steps and propose displaced patch parallelism, which takes advantage of the sequential nature of the diffusion process by reusing the pre-computed feature maps from the previous timestep to provide context for the current step. Therefore, our method supports asynchronous communication, which can be pipelined by computation. Extensive experiments show that our method can be applied to recent Stable Diffusion XL with no quality degradation and achieve up to a 6.1× speedup on eight NVIDIA A100s compared to one. Our code is publicly available at https://github.com/mit-han-lab/distrifuser.

Key facts. Cited by 98 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 64 of its 79 references that are in the map. First posted on arXiv 29 Feb 2024.

Builds on

64 of 79 references.

Closest ancestors · 23

Same direction, newest first

  1. 2023LCM-LoRA: A Universal Stable-Diffusion Acceleration Module9 Nov
  2. 2023Matryoshka Diffusion Models23 Oct
  3. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference6 Oct
  4. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  5. 2023Parallel Sampling of Diffusion Models25 May
  6. 2023Consistency Models2 Mar
  7. 2023Q-Diffusion: Quantizing Diffusion Models8 Feb
  8. 2023Simple diffusion: End-to-end diffusion for high resolution images26 Jan
  9. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models2 Nov
  10. 2022On Distillation of Guided Diffusion Models6 Oct
  11. 2022Classifier-Free Diffusion Guidance26 Jul
  12. 2022gDDIM: Generalized denoising diffusion implicit models11 Jun
11 more
  1. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  2. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  3. 2022Fast Sampling of Diffusion Models with Exponential Integrator29 Apr
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  5. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANs15 Dec
  6. 2021Score-based Generative Modeling in Latent Space10 Jun
  7. 2021On Fast Sampling of Diffusion Probabilistic Models31 May
  8. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  9. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  10. 2020Denoising Diffusion Implicit Models6 Oct
  11. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 41

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  9. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  10. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  11. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  12. 2018Group Normalizationcited by 1,514
29 more
  1. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  2. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  3. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  4. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  5. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 641
  6. 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 621
  7. 2016OctNet: Learning Deep 3D Representations at High Resolutionscited by 340
  8. 2021ZeRO-Offload: Democratizing Billion-Scale Model Trainingcited by 339
  9. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
  10. 2014Speeding up Convolutional Neural Networks with Low Rank Expansionscited by 283
  11. 2023Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystackcited by 278
  12. 2023SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Secondscited by 227
  13. 2018Beyond Data and Model Parallelism for Deep Neural Networkscited by 177
  14. 2021GSPMD: General and Scalable Parallelization for ML Computation Graphscited by 109
  15. 2023DiffCollage: Parallel Generation of Large Content with Diffusion Modelscited by 100
  16. 2021MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learningcited by 87
  17. 2021TeraPipe: Token-Level Pipeline Parallelism for Training Large-Scale Language Modelscited by 74
  18. 2022TorchSparse: Efficient Point Cloud Inference Enginecited by 72
  19. 2017More is Less: A More Complicated Network with Less Inference Complexitycited by 66
  20. 20223DDesigner: Towards Photorealistic 3D Object Generation and Editing with Text-guided Diffusion Modelscited by 57
  21. 2017Not All Pixels Are Equal: Difficulty-aware Semantic Segmentation via Deep Layer Cascadecited by 53
  22. 2018SBNet: Sparse Blocks Network for Fast Inferencecited by 53
  23. 2022Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Modelscited by 44
  24. 2023TorchSparse++: Efficient Training and Inference Framework for Sparse Convolution on GPUscited by 36
  25. 2023AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Servingcited by 31
  26. 2021OneFlow: Redesign the Distributed Deep Learning Framework from Scratchcited by 15
  27. 2018Recurrent Residual Module for Fast Inference in Videoscited by 13
  28. 2017Cnvlutin2: Ineffectual-Activation-and-Weight-Free Deep Neural Network Computingcited by 9
  29. 2017Speeding up Convolutional Neural Networks By Exploiting the Sparsity of Rectifier Unitscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 28; 2025: 42; 2026: 28.

98 papers.

Sort:

By area:

  1. 2024Timestep Embedding Tells: It's Time to Cache for Video Diffusion Modelcited by 245
  2. 2024Deep Compression Autoencoder for Efficient High-Resolution Diffusion Modelscited by 209
  3. 2025Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsitycited by 204
  4. 2024SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Modelscited by 161
  5. 2024Real-Time Video Generation with Pyramid Attention Broadcastcited by 140
  6. 2024Learning-to-Cache: Accelerating Diffusion Transformer via Layer Cachingcited by 138
  7. 2025XAttention: Block Sparse Attention with Antidiagonal Scoringcited by 131
  8. 2025Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutationcited by 100
  9. 2025Training-free and Adaptive Sparse Attention for Efficient Long Video Generationcited by 65
  10. 2025Radial Attention: O(nlog n) Sparse Attention with Energy Decay for Long Video Generationcited by 64
  11. 2024xDiT: an Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelismcited by 55
  12. 2024LinFusion: 1 GPU, 1 Minute, 16K Imagecited by 42
38 more
  1. 2025Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generationcited by 36
  2. 2025Efficient Diffusion Models: A Surveycited by 35
  3. 2025VMoBA: Mixture-of-Block Attention for Video Diffusion Modelscited by 35
  4. 2024Collaborative Decoding Makes Visual Auto-Regressive Modeling Efficientcited by 32
  5. 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
  6. 2025Accelerated Diffusion Models via Speculative Samplingcited by 30
  7. 2024Video-Infinity: Distributed Long Video Generationcited by 29
  8. 2024PipeFusion: Patch-level Pipeline Parallelism for Diffusion Transformers Inferencecited by 28
  9. 2024Diffusion Models Meet Remote Sensing: Principles, Methods, and Perspectivescited by 27
  10. 2025StreamDiffusionV2: A Streaming System for Dynamic and Interactive Video Generationcited by 27
  11. 2024CLEAR: Conv-Like Linearization Revs Pre-Trained Diffusion Transformers Upcited by 26
  12. 2025Efficient-vDiT: Efficient Video Diffusion Transformers With Attention Tilecited by 26
  13. 2024AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoisingcited by 22
  14. 2025DraftAttention: Fast Video Diffusion via Low-Resolution Attention Guidancecited by 21
  15. 2024A Survey on Diffusion Models for Recommender Systemscited by 20
  16. 2025FPSAttention: Training-Aware FP8 and Sparsity Co-Design for Fast Video Diffusioncited by 19
  17. 2025CAT Pruning: Cluster-Aware Token Pruning For Text-to-Image Diffusion Modelscited by 16
  18. 2024Is One GPU Enough? Pushing Image Generation at Higher-Resolutions with Foundation Modelscited by 14
  19. 2025A Survey on Cache Methods in Diffusion Models: Toward Efficient Multi-Modal Generationcited by 14
  20. 2024Training-Free Adaptive Diffusion with Bounded Difference Approximation Strategycited by 13
  21. 2025CacheQuant: Comprehensively Accelerated Diffusion Modelscited by 13
  22. 2025Distilling Parallel Gradients for Fast ODE Solvers of Diffusion Modelscited by 13
  23. 2025Re-ttention: Ultra Sparse Visual Generation via Attention Statistical Reshapecited by 10
  24. 2025LiT: Delving into a Simple Linear Diffusion Transformer for Image Generationcited by 9
  25. 2025Adaptive Diffusion Guidance via Stochastic Optimal Controlcited by 9
  26. 2025LeMiCa: Lexicographic Minimax Path Caching for Efficient Diffusion-Based Video Generationcited by 9
  27. 2026SVG-EAR: Parameter-Free Linear Compensation for Sparse Video Generation via Error-aware Routingcited by 9
  28. 2025SRDiffusion: Accelerate Video Diffusion Inference via Sketching-Rendering Cooperationcited by 7
  29. 2025DC-Gen: Post-Training Diffusion Acceleration with Deeply Compressed Latent Spacecited by 7
  30. 2026MeanCache: From Instantaneous to Average Velocity for Accelerating Flow Matching Inferencecited by 7
  31. 2024Parallel Simulation for Log-concave Sampling and Score-based Diffusion Modelscited by 6
  32. 2025SuperGen: An Efficient Ultra-high-resolution Video Generation System with Sketching and Tilingcited by 6
  33. 2026DFSAttn: Dynamic Fine-grained Sparse Attention for Efficient Video Generationcited by 6
  34. 2024SwiftDiffusion: Efficient Diffusion Model Serving with Add-on Modulescited by 5
  35. 2024AsCAN: Asymmetric Convolution-Attention Networks for Efficient Recognition and Generationcited by 5
  36. 2025One-Way Ticket:Time-Independent Unified Encoder for Distilling Text-to-Image Diffusion Modelscited by 5
  37. 2026Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithmscited by 5
  38. 2024FlexCache: Flexible Approximate Cache System for Video Diffusioncited by 4

50 most cited of 98 All 98 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexity
  2. 2026FastUSP: A Multi-Level Collaborative Acceleration Framework for Distributed Diffusion Model Inference
  3. 2025Minute-Long Videos with Dual Parallelisms
  4. 2025DiffPro: Joint Timestep and Layer-Wise Precision Optimization for Efficient Diffusion Inference
  5. 2025Paris: A Decentralized Trained Open-Weight Diffusion Model
  6. 2025Performance Plateaus in Inference-Time Scaling for Text-to-Image Diffusion Without External Models
  7. 2025CHORDS: Diffusion Sampling Accelerator with Multi-core Hierarchical ODE Solvers
  8. 2025Energy Scaling Laws for Diffusion Models: Quantifying Compute in Image Generation