# Speed Is All You Need: On-Device Acceleration of Large Diffusion Models via GPU-Aware Optimizations

arXiv: https://arxiv.org/abs/2304.11267 · HTML: https://mlsanity.com/paper/2304.11267/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 21 of its 29 references that are in the map.
- First posted on arXiv 21 Apr 2023.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Wasserstein GAN](https://mlsanity.com/paper/1701.07875/) (2017)
- [Fast Algorithms for Convolutional Neural Networks](https://mlsanity.com/paper/1509.09308/) (2015)
- [Efficient Memory Management for Deep Neural Net Inference](https://mlsanity.com/paper/2001.03288/) (2020)
- [Winograd Convolution for Deep Neural Networks: Efficient Point Selection](https://mlsanity.com/paper/2201.10369/) (2022)

## Built on by (most cited first, 30 of 45)

- [SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Seconds](https://mlsanity.com/paper/2306.00980/) (2023)
- [Frontier AI Regulation: Managing Emerging Risks to Public Safety](https://mlsanity.com/paper/2307.03718/) (2023)
- [Distilling Diffusion Models into Conditional GANs](https://mlsanity.com/paper/2405.05967/) (2024)
- [LAPTOP-Diff: Layer Pruning and Normalized Distillation for Compressing Diffusion Models](https://mlsanity.com/paper/2404.11098/) (2024)
- [MixDQ: Memory-Efficient Few-Step Text-to-Image Diffusion Models with Metric-Decoupled Mixed Precision Quantization](https://mlsanity.com/paper/2405.17873/) (2024)
- [Cut Your Losses in Large-Vocabulary Language Models](https://mlsanity.com/paper/2411.09009/) (2024)
- [Efficient Diffusion Models: A Survey](https://mlsanity.com/paper/2502.06805/) (2025)
- [Towards High-quality HDR Deghosting with Conditional Diffusion Models](https://mlsanity.com/paper/2311.00932/) (2023)
- [HiDiffusion: Unlocking Higher-Resolution Creativity and Efficiency in Pretrained Diffusion Models](https://mlsanity.com/paper/2311.17528/) (2023)
- [Diffusion Models Meet Remote Sensing: Principles, Methods, and Perspectives](https://mlsanity.com/paper/2404.08926/) (2024)
- [Phoenix: A Federated Generative Diffusion Model](https://mlsanity.com/paper/2306.04098/) (2023)
- [KOALA: Empirical Lessons Toward Memory-Efficient and Fast Diffusion Models for Text-to-Image Synthesis](https://mlsanity.com/paper/2312.04005/) (2023)
- [Clockwork Diffusion: Efficient Generation With Model-Step Distillation](https://mlsanity.com/paper/2312.08128/) (2023)
- [Mobile Video Diffusion](https://mlsanity.com/paper/2412.07583/) (2024)
- [Softmax Bias Correction for Quantized Generative Models](https://mlsanity.com/paper/2309.01729/) (2023)
- [On-device Sora: Enabling Training-Free Diffusion-based Text-to-Video Generation for Mobile Devices](https://mlsanity.com/paper/2502.04363/) (2025)
- [BK-SDM: A Lightweight, Fast, and Cheap Version of Stable Diffusion](https://mlsanity.com/paper/2305.15798/) (2023)
- [SmartMem: Layout Transformation Elimination and Adaptation for Efficient DNN Execution on Mobile](https://mlsanity.com/paper/2404.13528/) (2024)
- [Squeezing Large-Scale Diffusion Models for Mobile](https://mlsanity.com/paper/2307.01193/) (2023)
- [Generative Artificial Intelligence Assisted Wireless Sensing: Human Flow Detection in Practical Communication Environments](https://mlsanity.com/paper/2404.14140/) (2024)
- [Inference Optimization of Foundation Models on AI Accelerators](https://mlsanity.com/paper/2407.09111/) (2024)
- [Anomaly Detection and Generation with Diffusion Models: A Survey](https://mlsanity.com/paper/2506.09368/) (2025)
- [Towards an On-device Agent for Text Rewriting](https://mlsanity.com/paper/2308.11807/) (2023)
- [SELMA: A Speech-Enabled Language Model for Virtual Assistant Interactions](https://mlsanity.com/paper/2501.19377/) (2025)
- [SwiftDiffusion: Efficient Diffusion Model Serving with Add-on Modules](https://mlsanity.com/paper/2407.02031/) (2024)
- [LaSe-E2V: Towards Language-guided Semantic-Aware Event-to-Video Reconstruction](https://mlsanity.com/paper/2407.05547/) (2024)
- [Temporal Feature Matters: A Framework for Diffusion Model Quantization](https://mlsanity.com/paper/2407.19547/) (2024)
- [Random Conditioning with Distillation for Data-Efficient Diffusion Model Compression](https://mlsanity.com/paper/2504.02011/) (2025)
- [Weather-Magician: Reconstruction and Rendering Framework for 4D Weather Synthesis In Real Time](https://mlsanity.com/paper/2505.19919/) (2025)
- [Cost-Aware Routing for Efficient Text-To-Image Generation](https://mlsanity.com/paper/2506.14753/) (2025)

## Abstract (arXiv)

The rapid development and application of foundation models have revolutionized the field of artificial intelligence. Large diffusion models have gained significant attention for their ability to generate photorealistic images and support various tasks. On-device deployment of these models provides benefits such as lower server costs, offline functionality, and improved user privacy. However, common large diffusion models have over 1 billion parameters and pose challenges due to restricted computational and memory resources on devices. We present a series of implementation optimizations for large diffusion models that achieve the fastest reported inference latency to-date (under 12 seconds for Stable Diffusion 1.4 without int8 quantization on Samsung S23 Ultra for a 512x512 image with 20 iterations) on GPU-equipped mobile devices. These enhancements broaden the applicability of generative AI and improve the overall user experience across a wide range of devices.