# MixDQ: Memory-Efficient Few-Step Text-to-Image Diffusion Models with Metric-Decoupled Mixed Precision Quantization

arXiv: https://arxiv.org/abs/2405.17873 · HTML: https://mlsanity.com/paper/2405.17873/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 41 of its 54 references that are in the map.
- First posted on arXiv 28 May 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 41)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [Efficient Streaming Language Models with Attention Sinks](https://mlsanity.com/paper/2309.17453/) (2023)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis](https://mlsanity.com/paper/2310.00426/) (2023)
- [Adversarial Diffusion Distillation](https://mlsanity.com/paper/2311.17042/) (2023)
- [On Distillation of Guided Diffusion Models](https://mlsanity.com/paper/2210.03142/) (2022)
- [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020)
- [A White Paper on Neural Network Quantization](https://mlsanity.com/paper/2106.08295/) (2021)
- [UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models](https://mlsanity.com/paper/2302.04867/) (2023)
- [HAQ: Hardware-Aware Automated Quantization with Mixed Precision](https://mlsanity.com/paper/1811.08886/) (2018)
- [HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision](https://mlsanity.com/paper/1905.03696/) (2019)
- [Q-Diffusion: Quantizing Diffusion Models](https://mlsanity.com/paper/2302.04304/) (2023)
- [LCM-LoRA: A Universal Stable-Diffusion Acceleration Module](https://mlsanity.com/paper/2311.05556/) (2023)
- [FP8 Formats for Deep Learning](https://mlsanity.com/paper/2209.05433/) (2022)
- [PTQD: Accurate Post-Training Quantization for Diffusion Models](https://mlsanity.com/paper/2305.10657/) (2023)
- [Text-to-Text Pre-Training for Data-to-Text Tasks](https://mlsanity.com/paper/2005.10433/) (2020)
- [Temporal Dynamic Quantization for Diffusion Models](https://mlsanity.com/paper/2306.02316/) (2023)

## Built on by (most cited first, 30 of 43)

- [SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models](https://mlsanity.com/paper/2411.05007/) (2024)
- [DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMs](https://mlsanity.com/paper/2406.01721/) (2024)
- [SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration](https://mlsanity.com/paper/2410.02367/) (2024)
- [ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation](https://mlsanity.com/paper/2406.02540/) (2024)
- [DiTFastAttn: Attention Compression for Diffusion Transformer Models](https://mlsanity.com/paper/2406.08552/) (2024)
- [Accelerating Auto-regressive Text-to-Image Generation with Training-free Speculative Jacobi Decoding](https://mlsanity.com/paper/2410.01699/) (2024)
- [BitsFusion: 1.99 bits Weight Quantization of Diffusion Model](https://mlsanity.com/paper/2406.04333/) (2024)
- [1.58-bit FLUX](https://mlsanity.com/paper/2412.18653/) (2024)
- [Token Pruning for Caching Better: 9 Times Acceleration on Stable Diffusion for Free](https://mlsanity.com/paper/2501.00375/) (2024)
- [QuantVLA: Scale-Calibrated Post-Training Quantization for Vision-Language-Action Models](https://mlsanity.com/paper/2602.20309/) (2026)
- [Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMs](https://mlsanity.com/paper/2508.14896/) (2025)
- [HiFlow: Training-free High-Resolution Image Generation with Flow-Aligned Guidance](https://mlsanity.com/paper/2504.06232/) (2025)
- [PM-KVQ: Progressive Mixed-precision KV Cache Quantization for Long-CoT LLMs](https://mlsanity.com/paper/2505.18610/) (2025)
- [FPSAttention: Training-Aware FP8 and Sparsity Co-Design for Fast Video Diffusion](https://mlsanity.com/paper/2506.04648/) (2025)
- [PAROAttention: Pattern-Aware ReOrdering for Efficient Sparse and Quantized Attention in Visual Generation Models](https://mlsanity.com/paper/2506.16054/) (2025)
- [DGQ: Distribution-Aware Group Quantization for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2501.04304/) (2025)
- [Low-bit Model Quantization for Deep Neural Networks: A Survey](https://mlsanity.com/paper/2505.05530/) (2025)
- [ENAT: Rethinking Spatial-temporal Interactions in Token-based Image Synthesis](https://mlsanity.com/paper/2411.06959/) (2024)
- [MixDiT: Accelerating Image Diffusion Transformer Inference with Mixed-Precision MX Quantization](https://mlsanity.com/paper/2504.08398/) (2025)
- [ConvRot: Rotation-Based Plug-and-Play 4-bit Quantization for Diffusion Transformers](https://mlsanity.com/paper/2512.03673/) (2025)
- [Diffusion Model Quantization: A Review](https://mlsanity.com/paper/2505.05215/) (2025)
- [Modulated Diffusion: Accelerating Generative Modeling with Modulated Quantization](https://mlsanity.com/paper/2506.22463/) (2025)
- [Error Propagation Mechanisms and Compensation Strategies for Quantized Diffusion](https://mlsanity.com/paper/2508.12094/) (2025)
- [DiRotQ: Rotation-Aware Quantization for 4-bit Diffusion Transformers](https://mlsanity.com/paper/2605.16732/) (2026)
- [Q-Drift: Quantization-Aware Drift Correction for Diffusion Model Sampling](https://mlsanity.com/paper/2603.18095/) (2026)
- [Q-Sched: Pushing the Boundaries of Few-Step Diffusion Models with Quantization-Aware Scheduling](https://mlsanity.com/paper/2509.01624/) (2025)
- [TreeQ: Pushing the Quantization Boundary of Diffusion Transformer via Tree-Structured Mixed-Precision Search](https://mlsanity.com/paper/2512.06353/) (2025)
- [Adaptive Routing of Text-to-Image Generation Requests Between Large Cloud Model and Light-Weight Edge Model](https://mlsanity.com/paper/2411.13787/) (2024)
- [LightCache: Memory-Efficient, Training-Free Acceleration for Video Generation](https://mlsanity.com/paper/2510.05367/) (2025)
- [SJD++: Improved Speculative Jacobi Decoding for Training-free Acceleration of Discrete Auto-regressive Text-to-Image Generation](https://mlsanity.com/paper/2512.07503/) (2025)

## Abstract (arXiv)

Diffusion models have achieved significant visual generation quality. However, their significant computational and memory costs pose challenge for their application on resource-constrained mobile devices or even desktop GPUs. Recent few-step diffusion models reduces the inference time by reducing the denoising steps. However, their memory consumptions are still excessive. The Post Training Quantization (PTQ) replaces high bit-width FP representation with low-bit integer values (INT4/8) , which is an effective and efficient technique to reduce the memory cost. However, when applying to few-step diffusion models, existing quantization methods face challenges in preserving both the image quality and text alignment. To address this issue, we propose an mixed-precision quantization framework - MixDQ. Firstly, We design specialized BOS-aware quantization method for highly sensitive text embedding quantization. Then, we conduct metric-decoupled sensitivity analysis to measure the sensitivity of each layer. Finally, we develop an integer-programming-based method to conduct bit-width allocation. While existing quantization methods fall short at W8A8, MixDQ could achieve W8A8 without performance loss, and W4A8 with negligible visual degradation. Compared with FP16, we achieve 3-4x reduction in model size and memory cost, and 1.45x latency speedup.