mlsanityData to 7 Oct 2026

Efficient Quantization Strategies for Latent Diffusion Models

arXiv 2312.05431 ↗First posted 9 Dec 2023cs.CV cs.AIcited by 16

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Latent Diffusion Models (LDMs) capture the dynamic evolution of latent variables over time, blending patterns and multimodality in a generative system. Despite the proficiency of LDM in various applications, such as text-to-image generation, facilitated by robust text encoders and a variational autoencoder, the critical need to deploy large generative models on edge devices compels a search for more compact yet effective alternatives. Post Training Quantization (PTQ), a method to compress the operational size of deep learning models, encounters challenges when applied to LDM due to temporal and structural complexities. This study proposes a quantization strategy that efficiently quantize LDMs, leveraging Signal-to-Quantization-Noise Ratio (SQNR) as a pivotal metric for evaluation. By treating the quantization discrepancy as relative noise and identifying sensitive part(s) of a model, we propose an efficient quantization approach encompassing both global and local strategies. The global quantization process mitigates relative quantization noise by initiating higher-precision quantization on sensitive blocks, while local treatments address specific challenges in quantization-sensitive and time-sensitive modules. The outcomes of our experiments reveal that the implementation of both global and local treatments yields a highly efficient and effective Post Training Quantization (PTQ) of LDMs.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 42 of its 50 references that are in the map. First posted on arXiv 9 Dec 2023.

Builds on

42 of 50 references.

Closest ancestors · 16

Same direction, newest first

  1. 2023Softmax Bias Correction for Quantized Generative Models4 Sep
  2. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  3. 2023Towards Accurate Post-training Quantization for Diffusion Models30 May
  4. 2023Structural Pruning for Diffusion Models18 May
  5. 2023PTQD: Accurate Post-Training Quantization for Diffusion Models18 May
  6. 2023Token Merging for Fast Stable Diffusion30 Mar
  7. 2023Consistency Models2 Mar
  8. 2023Q-Diffusion: Quantizing Diffusion Models8 Feb
  9. 2022Post-training Quantization on Diffusion Models28 Nov
  10. 2022On Distillation of Guided Diffusion Models6 Oct
  11. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  12. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
4 more
  1. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesis29 Nov
  2. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  3. 2020Denoising Diffusion Implicit Models6 Oct
  4. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 26

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  5. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  6. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  7. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 640
  8. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 507
  9. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  10. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 426
  11. 2020Reliable Fidelity and Diversity Metrics for Generative Modelscited by 425
  12. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 387
14 more
  1. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 272
  2. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 265
  3. 2022BBDM: Image-to-image Translation with Brownian Bridge Diffusion Modelscited by 222
  4. 2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 176
  5. 2019Effectively Unbiased FID and Inception Score and where to find themcited by 161
  6. 2019Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networkscited by 143
  7. 2020EasyQuant: Post-training Quantization via Scale Optimizationcited by 57
  8. 2021Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimizationcited by 57
  9. 2020Differentiable Joint Pruning and Quantization for Hardware Efficiencycited by 45
  10. 2022Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Modelscited by 44
  11. 2021Towards Efficient Post-training Quantization of Pre-trained Language Modelscited by 37
  12. 2023A Practical Mixed Precision Algorithm for Post-Training Quantizationcited by 23
  13. 2018A Quantization-Friendly Separable Convolution for MobileNetscited by 22
  14. 2018Quantization for Rapid Deployment of Deep Neural Networkscited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 5; 2026: 2.

16 papers.

Sort:

By area:

  1. 2024SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Modelscited by 161
  2. 2024ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generationcited by 110
  3. 2024QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuningcited by 48
  4. 2024MixDQ: Memory-Efficient Few-Step Text-to-Image Diffusion Models with Metric-Decoupled Mixed Precision Quantizationcited by 42
  5. 2024BitsFusion: 1.99 bits Weight Quantization of Diffusion Modelcited by 37
  6. 20241.58-bit FLUXcited by 31
  7. 2024Not All Prompts Are Made Equal: Prompt-based Pruning of Text-to-Image Diffusion Modelscited by 13
  8. 2024TerDiT: Ternary Diffusion Models with Transformerscited by 6
  9. 2025Diffusion Model Quantization: A Reviewcited by 6
  10. 2025Error Propagation Mechanisms and Compensation Strategies for Quantized Diffusioncited by 5
  11. 2025Diffusion on Graph: Augmentation of Graph Structure for Node Classificationcited by 4
  12. 2024Accurate Compression of Text-to-Image Diffusion Models via Vector Quantizationcited by 3
4 more
  1. 2025SQ-DM: Accelerating Diffusion Models with Aggressive Quantization and Temporal Sparsitycited by 3
  2. 2026LoRaQ: Optimized Low Rank Approximation for 4-bit Quantizationcited by 2
  3. 2025Hardware-Friendly Static Quantization Method for Video Diffusion Transformerscited by 1
  4. 2026A KL Lens on Quantization: Fast, Forward-Only Sensitivity for Mixed-Precision SSM-Transformer Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2023EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models
  2. 2024Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers
  3. 2024PQD: Post-training Quantization for Efficient Diffusion Models
  4. 2025DMQ: Dissecting Outliers of Diffusion Models for Post-Training Quantization
  5. 2024Efficiency Meets Fidelity: A Novel Quantization Framework for Stable Diffusion
  6. 2024DilateQuant: Accurate and Efficient Diffusion Quantization via Weight Dilation
  7. 2024MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Models
  8. 2025SegQuant: A Semantics-Aware and Generalizable Quantization Framework for Diffusion Models