mlsanityData to 7 Oct 2026

Speed Is All You Need: On-Device Acceleration of Large Diffusion Models via GPU-Aware Optimizations

arXiv 2304.11267 ↗First posted 21 Apr 2023cs.CV cs.LG eess.IVcited by 45

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

The rapid development and application of foundation models have revolutionized the field of artificial intelligence. Large diffusion models have gained significant attention for their ability to generate photorealistic images and support various tasks. On-device deployment of these models provides benefits such as lower server costs, offline functionality, and improved user privacy. However, common large diffusion models have over 1 billion parameters and pose challenges due to restricted computational and memory resources on devices. We present a series of implementation optimizations for large diffusion models that achieve the fastest reported inference latency to-date (under 12 seconds for Stable Diffusion 1.4 without int8 quantization on Samsung S23 Ultra for a 512x512 image with 20 iterations) on GPU-equipped mobile devices. These enhancements broaden the applicability of generative AI and improve the overall user experience across a wide range of devices.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 21 of its 29 references that are in the map. First posted on arXiv 21 Apr 2023.

Builds on

21 of 29 references.

Closest ancestors · 2

Same direction, newest first

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  2. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2022Generative Adversarial Networkscited by 5,603
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  6. 2013Auto-Encoding Variational Bayescited by 3,980
  7. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  8. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  9. 2020Longformer: The Long-Document Transformercited by 3,362
  10. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  11. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  12. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
7 more
  1. 2020Reformer: The Efficient Transformercited by 1,658
  2. 2020Rethinking Attention with Performerscited by 1,584
  3. 2018Group Normalizationcited by 1,514
  4. 2017Wasserstein GANcited by 635
  5. 2015Fast Algorithms for Convolutional Neural Networkscited by 102
  6. 2020Efficient Memory Management for Deep Neural Net Inferencecited by 12
  7. 2022Winograd Convolution for Deep Neural Networks: Efficient Point Selectioncited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 11; 2024: 14; 2025: 16; 2026: 4.

45 papers.

Sort:

By area:

  1. 2023SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Secondscited by 227
  2. 2023Frontier AI Regulation: Managing Emerging Risks to Public Safetycited by 95
  3. 2024Distilling Diffusion Models into Conditional GANscited by 78
  4. 2024LAPTOP-Diff: Layer Pruning and Normalized Distillation for Compressing Diffusion Modelscited by 52
  5. 2024MixDQ: Memory-Efficient Few-Step Text-to-Image Diffusion Models with Metric-Decoupled Mixed Precision Quantizationcited by 42
  6. 2024Cut Your Losses in Large-Vocabulary Language Modelscited by 35
  7. 2025Efficient Diffusion Models: A Surveycited by 35
  8. 2023Towards High-quality HDR Deghosting with Conditional Diffusion Modelscited by 33
  9. 2023HiDiffusion: Unlocking Higher-Resolution Creativity and Efficiency in Pretrained Diffusion Modelscited by 27
  10. 2024Diffusion Models Meet Remote Sensing: Principles, Methods, and Perspectivescited by 27
  11. 2023Phoenix: A Federated Generative Diffusion Modelcited by 21
  12. 2023KOALA: Empirical Lessons Toward Memory-Efficient and Fast Diffusion Models for Text-to-Image Synthesiscited by 18
33 more
  1. 2023Clockwork Diffusion: Efficient Generation With Model-Step Distillationcited by 18
  2. 2024Mobile Video Diffusioncited by 17
  3. 2023Softmax Bias Correction for Quantized Generative Modelscited by 15
  4. 2025On-device Sora: Enabling Training-Free Diffusion-based Text-to-Video Generation for Mobile Devicescited by 14
  5. 2023BK-SDM: A Lightweight, Fast, and Cheap Version of Stable Diffusioncited by 13
  6. 2024SmartMem: Layout Transformation Elimination and Adaptation for Efficient DNN Execution on Mobilecited by 12
  7. 2023Squeezing Large-Scale Diffusion Models for Mobilecited by 11
  8. 2024Generative Artificial Intelligence Assisted Wireless Sensing: Human Flow Detection in Practical Communication Environmentscited by 9
  9. 2024Inference Optimization of Foundation Models on AI Acceleratorscited by 9
  10. 2025Anomaly Detection and Generation with Diffusion Models: A Surveycited by 8
  11. 2023Towards an On-device Agent for Text Rewritingcited by 7
  12. 2025SELMA: A Speech-Enabled Language Model for Virtual Assistant Interactionscited by 6
  13. 2024SwiftDiffusion: Efficient Diffusion Model Serving with Add-on Modulescited by 5
  14. 2024LaSe-E2V: Towards Language-guided Semantic-Aware Event-to-Video Reconstructioncited by 5
  15. 2024Temporal Feature Matters: A Framework for Diffusion Model Quantizationcited by 5
  16. 2025Random Conditioning with Distillation for Data-Efficient Diffusion Model Compressioncited by 5
  17. 2025Cost-Aware Routing for Efficient Text-To-Image Generationcited by 4
  18. 2024A-SDM: Accelerating Stable Diffusion through Model Assembly and Feature Inheritance Strategiescited by 3
  19. 2025Scaling On-Device GPU Inference for Large Generative Modelscited by 3
  20. 2025Weather-Magician: Reconstruction and Rendering Framework for 4D Weather Synthesis In Real Timecited by 3
  21. 2026Multi-Modal Data-Enhanced Foundation Models for Prediction and Control in Wireless Networks: A Surveycited by 3
  22. 2025Efficient High-Resolution Image Editing with Hallucination-Aware Loss and Adaptive Tilingcited by 2
  23. 2024Non-equilibrium active noise enhances generative memory in diffusion modelscited by 1
  24. 2025Stealing Creator's Workflow: A Creator-Inspired Agentic Framework with Iterative Feedback Loop for Improved Scientific Short-form Generationcited by 1
  25. 2025DiTOX: Fault Detection and Localization in the ONNX Optimizercited by 1
  26. 2025Generative AI Meets 6G and Beyond: Diffusion Models for Semantic Communicationscited by 1
  27. 2025Intelligent Mobile AI-Generated Content Services via Interactive Prompt Engineering and Dynamic Service Provisioningcited by 0
  28. 2025PromptMobile: Efficient Promptus for Low Bandwidth Mobile Video Streamingcited by 0
  29. 2025On-device Sora: Enabling Training-Free Diffusion-based Text-to-Video Generation for Mobile Devicescited by 0
  30. 2025SP-Guard: Selective Prompt-adaptive Guidance for Safe Text-to-Image Generationcited by 0
  31. 2026SODA: Sensitivity-Oriented Dynamic Acceleration for Diffusion Transformercited by 0
  32. 2026Accelerating Diffusion Models for Generative AI Applications with Silicon Photonicscited by 0
  33. 2026OFA-Diffusion Compression: Compressing Diffusion Model in One-Shot Mannercited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026FastUSP: A Multi-Level Collaborative Acceleration Framework for Distributed Diffusion Model Inference
  2. 2025Energy Scaling Laws for Diffusion Models: Quantifying Compute in Image Generation
  3. 2026Systematic Optimization of Real-Time Diffusion Model Inference on Apple M3 Ultra
  4. 2025SQ-DM: Accelerating Diffusion Models with Aggressive Quantization and Temporal Sparsity
  5. 2024Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexity
  6. 2025Performance Plateaus in Inference-Time Scaling for Text-to-Image Diffusion Without External Models
  7. 2024DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models
  8. 2024SparseDM: Toward Sparse Efficient Diffusion Models