# MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Models

arXiv: https://arxiv.org/abs/2412.11549 · HTML: https://mlsanity.com/paper/2412.11549/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 33 of its 46 references that are in the map.
- First posted on arXiv 16 Dec 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 33)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [SpinQuant: LLM quantization with learned rotations](https://mlsanity.com/paper/2405.16406/) (2024)
- [HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision](https://mlsanity.com/paper/1905.03696/) (2019)
- [Post-training Quantization on Diffusion Models](https://mlsanity.com/paper/2211.15736/) (2022)
- [Generating Images with Sparse Representations](https://mlsanity.com/paper/2103.03841/) (2021)
- [HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks](https://mlsanity.com/paper/1911.03852/) (2019)
- [ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions](https://mlsanity.com/paper/2003.03488/) (2020)
- [QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantization](https://mlsanity.com/paper/2203.05740/) (2022)
- [PTQD: Accurate Post-Training Quantization for Diffusion Models](https://mlsanity.com/paper/2305.10657/) (2023)
- [Forward and Backward Information Retention for Accurate Binary Neural Networks](https://mlsanity.com/paper/1909.10788/) (2019)
- [Cross-Image Relational Knowledge Distillation for Semantic Segmentation](https://mlsanity.com/paper/2204.06986/) (2022)
- [ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation](https://mlsanity.com/paper/2406.02540/) (2024)
- [Training Binary Neural Networks with Real-to-Binary Convolutions](https://mlsanity.com/paper/2003.11535/) (2020)
- [Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming](https://mlsanity.com/paper/2006.10518/) (2020)
- [CLIP-KD: An Empirical Study of CLIP Model Distillation](https://mlsanity.com/paper/2307.12732/) (2023)
- [EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models](https://mlsanity.com/paper/2310.03270/) (2023)
- [PD-Quant: Post-Training Quantization based on Prediction Difference Metric](https://mlsanity.com/paper/2212.07048/) (2022)
- [TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models](https://mlsanity.com/paper/2311.16503/) (2023)
- [QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuning](https://mlsanity.com/paper/2402.03666/) (2024)
- [Towards Accurate Post-training Quantization for Diffusion Models](https://mlsanity.com/paper/2305.18723/) (2023)
- [Online Knowledge Distillation via Mutual Contrastive Learning for Visual Recognition](https://mlsanity.com/paper/2207.11518/) (2022)
- [A Survey of Techniques for Optimizing Transformer Inference](https://mlsanity.com/paper/2307.07982/) (2023)

## Built on by (most cited first, 21 of 21)

- [Quantized Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2509.21302/) (2025)
- [Accelerating Diffusion Transformer via Error-Optimized Cache](https://mlsanity.com/paper/2501.19243/) (2025)
- [MPQ-DMv2: Flexible Residual Mixed Precision Quantization for Low-Bit Diffusion Models with Temporal Distillation](https://mlsanity.com/paper/2507.04290/) (2025)
- [Low-bit Model Quantization for Deep Neural Networks: A Survey](https://mlsanity.com/paper/2505.05530/) (2025)
- [PersonaLive! Expressive Portrait Image Animation for Live Streaming](https://mlsanity.com/paper/2512.11253/) (2025)
- [FP4DiT: Towards Effective Floating Point Quantization for Diffusion Transformers](https://mlsanity.com/paper/2503.15465/) (2025)
- [Multi-party Collaborative Attention Control for Image Customization](https://mlsanity.com/paper/2505.01428/) (2025)
- [WorldCache: Accelerating World Models for Free via Heterogeneous Token Caching](https://mlsanity.com/paper/2603.06331/) (2026)
- [Modulated Diffusion: Accelerating Generative Modeling with Modulated Quantization](https://mlsanity.com/paper/2506.22463/) (2025)
- [Test-Time Iterative Error Correction for Efficient Diffusion Models](https://mlsanity.com/paper/2511.06250/) (2025)
- [TreeQ: Pushing the Quantization Boundary of Diffusion Transformer via Tree-Structured Mixed-Precision Search](https://mlsanity.com/paper/2512.06353/) (2025)
- [Fast-SAM3D: 3Dfy Anything in Images but Faster](https://mlsanity.com/paper/2602.05293/) (2026)
- [SSVQ: Unleashing the Potential of Vector Quantization with Sign-Splitting](https://mlsanity.com/paper/2503.08668/) (2025)
- [HQ-DM: Single Hadamard Transformation-Based Quantization-Aware Training for Low-Bit Diffusion Models](https://mlsanity.com/paper/2512.05746/) (2025)
- [Gradient-Aligned Calibration for Post-Training Quantization of Diffusion Models](https://mlsanity.com/paper/2602.01289/) (2026)
- [TMPDiff: Temporal Mixed-Precision for Diffusion Models](https://mlsanity.com/paper/2603.14062/) (2026)
- [TASQ: Temporal-Adaptive Bit Sparsification Quantization for Diffusion Models](https://mlsanity.com/paper/2608.03057/) (2026)
- [EditaLive! Unified Character Video Editing for Live Streaming](https://mlsanity.com/paper/2608.27123/) (2026)
- [Same Bit Width, Different Outcomes: Post-Training Quantization of Text-to-Speech Across Architectures](https://mlsanity.com/paper/2609.28974/) (2026)
- [Precision As You Need: Stochastic Computing Is a Dense Adaptive Quantizer](https://mlsanity.com/paper/2609.32922/) (2026)
- [ManifoldCache: Training-Free Diffusion Acceleration via Constraint Manifold Caching](https://mlsanity.com/paper/2610.04510/) (2026)

## Abstract (arXiv)

Diffusion models have received wide attention in generation tasks. However, the expensive computation cost prevents the application of diffusion models in resource-constrained scenarios. Quantization emerges as a practical solution that significantly saves storage and computation by reducing the bit-width of parameters. However, the existing quantization methods for diffusion models still cause severe degradation in performance, especially under extremely low bit-widths (2-4 bit). The primary decrease in performance comes from the significant discretization of activation values at low bit quantization. Too few activation candidates are unfriendly for outlier significant weight channel quantization, and the discretized features prevent stable learning over different time steps of the diffusion model. This paper presents MPQ-DM, a Mixed-Precision Quantization method for Diffusion Models. The proposed MPQ-DM mainly relies on two techniques:(1) To mitigate the quantization error caused by outlier severe weight channels, we propose an Outlier-Driven Mixed Quantization (OMQ) technique that uses Kurtosis to quantify outlier salient channels and apply optimized intra-layer mixed-precision bit-width allocation to recover accuracy performance within target efficiency.(2) To robustly learn representations crossing time steps, we construct a Time-Smoothed Relation Distillation (TRD) scheme between the quantized diffusion model and its full-precision counterpart, transferring discrete and continuous latent to a unified relation space to reduce the representation inconsistency. Comprehensive experiments demonstrate that MPQ-DM achieves significant accuracy gains under extremely low bit-widths compared with SOTA quantization methods. MPQ-DM achieves a 58% FID decrease under W2A4 setting compared with baseline, while all other methods even collapse.