mlsanityData to 7 Oct 2026

SQ-DM: Accelerating Diffusion Models with Aggressive Quantization and Temporal Sparsity

arXiv 2501.15448 ↗First posted 26 Jan 2025cs.CV cs.AI cs.ARcited by 3

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models have gained significant popularity in image generation tasks. However, generating high-quality content remains notably slow because it requires running model inference over many time steps. To accelerate these models, we propose to aggressively quantize both weights and activations, while simultaneously promoting significant activation sparsity. We further observe that the stated sparsity pattern varies among different channels and evolves across time steps. To support this quantization and sparsity scheme, we present a novel diffusion model accelerator featuring a heterogeneous mixed-precision dense-sparse architecture, channel-last address mapping, and a time-step-aware sparsity detector for efficient handling of the sparsity pattern. Our 4-bit quantization technique demonstrates superior generation quality compared to existing 4-bit methods. Our custom accelerator achieves 6.91x speed-up and 51.5% energy reduction compared to traditional dense accelerators.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 25 of its 40 references that are in the map. First posted on arXiv 26 Jan 2025.

Builds on

25 of 40 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models7 Nov
  2. 2024SparseDM: Toward Sparse Efficient Diffusion Models16 Apr
  3. 2024TMPQ-DM: Joint Timestep Reduction and Quantization Precision Selection for Efficient Diffusion Models15 Apr
  4. 2023Efficient Quantization Strategies for Latent Diffusion Models9 Dec
  5. 2023Analyzing and Improving the Training Dynamics of Diffusion Models5 Dec
  6. 2023TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models27 Nov
  7. 2023Structural Pruning for Diffusion Models18 May
  8. 2023PTQD: Accurate Post-Training Quantization for Diffusion Models18 May
  9. 2023Q-Diffusion: Quantizing Diffusion Models8 Feb
  10. 2022Post-training Quantization on Diffusion Models28 Nov
  11. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun

Other references · 14

  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  2. 2022Video Diffusion Modelscited by 2,280
  3. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  4. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
  5. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
  6. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  7. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
  8. 2016EIE: Efficient Inference Engine on Compressed Deep Neural Networkcited by 291
  9. 2021Accelerating Sparse Deep Neural Networkscited by 213
  10. 2023Microscaling Data Formats for Deep Learningcited by 132
  11. 2021VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inferencecited by 59
  12. 2022Enabling Flexibility for Sparse Tensor Acceleration via Heterogeneitycited by 4
2 more
  1. 2021Griffin: Rethinking Sparse Optimization for Deep Learning Architecturescited by 1
  2. 2023The Sparsity Roofline: Understanding the Hardware Limits of Sparse Neural Networkscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Test-Time Iterative Error Correction for Efficient Diffusion Modelscited by 4
  2. 2026DDiT: Dynamic Patch Scheduling for Efficient Diffusion Transformerscited by 3
  3. 2026Realizing Native INT8 Compute for Diffusion Transformers on Consumer GPUs: A Fused INT8 GEMM Kernel for Ideogram 4.0cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026RT-Lynx: Putting GEMM Sparsity in the Right Place for Diffusion Models
  2. 2024QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuning
  3. 2023EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models
  4. 2024BiDM: Pushing the Limit of Quantization for Diffusion Models
  5. 2024Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers
  6. 20266Bit-Diffusion: Inference-Time Mixed-Precision Quantization for Video Diffusion Models
  7. 2026DiffSparse: Accelerating Diffusion Transformers with Learned Token Sparsity
  8. 2025Q-Sched: Pushing the Boundaries of Few-Step Diffusion Models with Quantization-Aware Scheduling