mlsanityData to 7 Oct 2026

BitsFusion: 1.99 bits Weight Quantization of Diffusion Model

arXiv 2406.04333 ↗First posted 6 Jun 2024cs.CVcited by 37

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion-based image generation models have achieved great success in recent years by showing the capability of synthesizing high-quality content. However, these models contain a huge number of parameters, resulting in a significantly large model size. Saving and transferring them is a major bottleneck for various applications, especially those running on resource-constrained devices. In this work, we develop a novel weight quantization method that quantizes the UNet from Stable Diffusion v1.5 to 1.99 bits, achieving a model with 7.9X smaller size while exhibiting even better generation quality than the original one. Our approach includes several novel techniques, such as assigning optimal bits to each layer, initializing the quantized model for better performance, and improving the training strategy to dramatically reduce quantization error. Furthermore, we extensively evaluate our quantized model across various benchmark datasets and through human evaluation to demonstrate its superior generation quality.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 74 of its 89 references that are in the map. First posted on arXiv 6 Jun 2024.

Builds on

74 of 89 references.

Closest ancestors · 23

Same direction, newest first

  1. 2024MixDQ: Memory-Efficient Few-Step Text-to-Image Diffusion Models with Metric-Decoupled Mixed Precision Quantization28 May
  2. 2024QNCD: Quantization Noise Correction for Diffusion Models28 Mar
  3. 2024QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuning6 Feb
  4. 2024EDA-DM: Enhanced Distribution Alignment for Post-Training Quantization of Diffusion Models9 Jan
  5. 2023Efficient Quantization Strategies for Latent Diffusion Models9 Dec
  6. 2023TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models27 Nov
  7. 2023Post-training Quantization for Text-to-Image Diffusion Models with Progressive Calibration and Activation Relaxing10 Nov
  8. 2023Effective Quantization for Diffusion Models on CPUs2 Nov
  9. 2023EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models5 Oct
  10. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  11. 2023Temporal Dynamic Quantization for Diffusion Models4 Jun
  12. 2023PTQD: Accurate Post-Training Quantization for Diffusion Models18 May
11 more
  1. 2023Q-Diffusion: Quantizing Diffusion Models8 Feb
  2. 2022Post-training Quantization on Diffusion Models28 Nov
  3. 2022Classifier-Free Diffusion Guidance26 Jul
  4. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  5. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  6. 2022Pseudo Numerical Methods for Diffusion Models on Manifolds20 Feb
  7. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  8. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  9. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  10. 2020Denoising Diffusion Implicit Models6 Oct
  11. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 51

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  10. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  11. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  12. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
38 more
  1. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  2. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  3. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  4. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  5. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  6. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  7. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  8. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  9. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  10. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  11. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
  12. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
  13. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  14. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  15. 2019Learned Step Size Quantizationcited by 604
  16. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  17. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 552
  18. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  19. 2021A White Paper on Neural Network Quantizationcited by 453
  20. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 450
  21. 2024SpinQuant: LLM quantization with learned rotationscited by 446
  22. 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 416
  23. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  24. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 328
  25. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 320
  26. 2023Photorealistic Video Generation with Diffusion Modelscited by 304
  27. 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 289
  28. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  29. 2022DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Modelscited by 230
  30. 2023SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Secondscited by 227
  31. 2016Trained Ternary Quantizationcited by 203
  32. 2016Ternary Weight Networkscited by 195
  33. 2022SINE: SINgle Image Editing with Text-to-Image Diffusion Modelscited by 159
  34. 2024Snap Video: Scaled Spatiotemporal Transformers for Text-to-Video Synthesiscited by 105
  35. 2021Network Quantization with Element-wise Gradient Scalingcited by 82
  36. 2023HyperHuman: Hyper-Realistic Human Generation with Latent Structural Diffusioncited by 68
  37. 2024TextCraftor: Your Text Encoder Can be Image Quality Controllercited by 27
  38. 2022F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantizationcited by 26

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 12; 2025: 19; 2026: 6.

37 papers.

Sort:

By area:

  1. 2024SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Modelscited by 161
  2. 2024FasterCache: Training-Free Video Diffusion Model Acceleration with High Qualitycited by 115
  3. 2025SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillationcited by 85
  4. 2025TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Modelcited by 57
  5. 2024TimeDiT: General-purpose Diffusion Transformers for Time Series Foundation Modelcited by 33
  6. 20241.58-bit FLUXcited by 31
  7. 2024SnapGen: Taming High-Resolution Text-to-Image Models for Mobile Devices with Efficient Architectures and Trainingcited by 27
  8. 202570% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float (DFloat11)cited by 23
  9. 2025A Survey of Token Compression for Efficient Multimodal Large Language Modelscited by 22
  10. 2025DGQ: Distribution-Aware Group Quantization for Text-to-Image Diffusion Modelscited by 11
  11. 2025Low-bit Model Quantization for Deep Neural Networks: A Surveycited by 11
  12. 2025FP4DiT: Towards Effective Floating Point Quantization for Diffusion Transformerscited by 6
25 more
  1. 2025Diffusion Model Quantization: A Reviewcited by 6
  2. 2025DMQ: Dissecting Outliers of Diffusion Models for Post-Training Quantizationcited by 6
  3. 2024Temporal Feature Matters: A Framework for Diffusion Model Quantizationcited by 5
  4. 2024FADA: Fast Diffusion Avatar Synthesis with Mixed-Supervised Multi-CFG Distillationcited by 4
  5. 2025WhisperKit: On-device Real-time ASR with Billion-Scale Transformerscited by 4
  6. 2025Are Image-to-Video Models Good Zero-Shot Image Editors?cited by 4
  7. 2024Efficiency Meets Fidelity: A Novel Quantization Framework for Stable Diffusioncited by 3
  8. 2025Consistent and Controllable Image Animation with Motion Linear Diffusion Transformerscited by 3
  9. 2026SnapGen++: Unleashing Diffusion Transformers for Efficient High-Fidelity Image Generation on Edge Devicescited by 3
  10. 2025Stabilizing Quantization-Aware Training by Implicit-Regularization on Hessian Matrixcited by 2
  11. 2025FraQAT: Quantization Aware Training with Fractional bitscited by 2
  12. 2024DilateQuant: Accurate and Efficient Diffusion Quantization via Weight Dilationcited by 1
  13. 2024Data-Free Group-Wise Fully Quantized Winograd Convolution via Learnable Scalescited by 1
  14. 2025Efficiently Training A Flat Neural Network Before It has been Quantizatedcited by 1
  15. 2026HyperVAttention: Efficient Sparse Attention with Spatio-Temporal Clustering for Video Diffusioncited by 1
  16. 2024MetaGreen: Meta-Learning Inspired Transformer Selection for Green Semantic Communicationcited by 0
  17. 2024Fine-Tuning Stable Diffusion XL for Stylistic Icon Generation: A Comparison of Caption Sizecited by 0
  18. 2025QArtSR: Quantization via Reverse-Module and Timestep-Retraining in One-Step Diffusion based Image Super-Resolutioncited by 0
  19. 2025QuantFace: Efficient Quantization for Face Restorationcited by 0
  20. 2025AQUATIC-Diff: Additive Quantization for Truly Tiny Compressed Diffusion Modelscited by 0
  21. 2025LowDiff: Efficient Diffusion Sampling with Low-Resolution Conditioncited by 0
  22. 2026Na"ive PAINE: Lightweight Text-to-Image Generation Improvement with Prompt Evaluationcited by 0
  23. 2026RotateAttention: RoPE-Aware Rotation and Range Rectification for INT4 Quantized Attention in Video Generationcited by 0
  24. 2026TASQ: Temporal-Adaptive Bit Sparsification Quantization for Diffusion Modelscited by 0
  25. 2026DSAQuant: Denoising-Stage-Aligned Quantization-Aware Training for Video Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024BiDM: Pushing the Limit of Quantization for Diffusion Models
  2. 2025Bridging the Gap between Gaussian Diffusion Models and Universal Quantization for Image Compression
  3. 2024Low-Bitwidth Floating Point Quantization for Efficient High-Quality Diffusion Models
  4. 2024Diffusion Product Quantization
  5. 2025SQ-DM: Accelerating Diffusion Models with Aggressive Quantization and Temporal Sparsity
  6. 2024MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Models
  7. 2024PQD: Post-training Quantization for Efficient Diffusion Models
  8. 20266Bit-Diffusion: Inference-Time Mixed-Precision Quantization for Video Diffusion Models