mlsanityData to 14 Oct 2026

HQ-DM: Single Hadamard Transformation-Based Quantization-Aware Training for Low-Bit Diffusion Models

arXiv 2512.05746 ↗First posted 5 Dec 2025cs.CVcited by 0

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models have demonstrated significant applications in the field of image generation. However, their high computational and memory costs pose challenges for deployment. Model quantization has emerged as a promising solution to reduce storage overhead and accelerate inference. Nevertheless, existing quantization methods for diffusion models struggle to mitigate outliers in activation matrices during inference, leading to substantial performance degradation under low-bit quantization scenarios. To address this, we propose HQ-DM, a novel Quantization-Aware Training framework that applies Single Hadamard Transformation to activation matrices. This approach effectively reduces activation outliers while preserving model performance under quantization. Compared to traditional Double Hadamard Transformation, our proposed scheme offers distinct advantages by seamlessly supporting INT convolution operations while preventing the amplification of weight outliers. For conditional generation on the ImageNet 256x256 dataset using the LDM-4 model, our W4A4 and W4A3 quantization schemes improve the Inception Score by 12.8% and 467.73%, respectively, over the existing state-of-the-art method.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 59 of its 75 references that are in the map. First posted on arXiv 5 Dec 2025.

Builds on

59 of 75 references.

Closest ancestors · 22

Same direction, newest first

  1. 2025DMQ: Dissecting Outliers of Diffusion Models for Post-Training Quantization17 Jul
  2. 2025Modulated Diffusion: Accelerating Generative Modeling with Modulated Quantization18 Jun
  3. 2025DGQ: Distribution-Aware Group Quantization for Text-to-Image Diffusion Models8 Jan
  4. 2024MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Models16 Dec
  5. 2024Learnable Sparsity for Vision Generative Models3 Dec
  6. 2024QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuning6 Feb
  7. 2023One-step Diffusion with Distribution Matching Distillation30 Nov
  8. 2023TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models27 Nov
  9. 2023EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models5 Oct
  10. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  11. 2023Temporal Dynamic Quantization for Diffusion Models4 Jun
  12. 2023Towards Accurate Post-training Quantization for Diffusion Models30 May
10 more
  1. 2023Structural Pruning for Diffusion Models18 May
  2. 2023PTQD: Accurate Post-Training Quantization for Diffusion Models18 May
  3. 2023Q-Diffusion: Quantizing Diffusion Models8 Feb
  4. 2022Scalable Diffusion Models with Transformers19 Dec
  5. 2022Post-training Quantization on Diffusion Models28 Nov
  6. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  7. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  8. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  9. 2020Denoising Diffusion Implicit Models6 Oct
  10. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 37

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  8. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  9. 2016Improved Techniques for Training GANscited by 2,970
  10. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  11. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  12. 2022Video Diffusion Modelscited by 2,305
25 more
  1. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  2. 2021Image Super-Resolution via Iterative Refinementcited by 1,511
  3. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  4. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,355
  5. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 927
  6. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  7. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 843
  8. 2016Pixel Recurrent Neural Networkscited by 816
  9. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  10. 2019Learned Step Size Quantizationcited by 628
  11. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
  12. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 521
  13. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  14. 2021SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Modelscited by 415
  15. 2023ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shiftingcited by 365
  16. 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretrainingcited by 344
  17. 2023BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusioncited by 282
  18. 2023A Survey on Model Compression for Large Language Modelscited by 269
  19. 2021Generating Images with Sparse Representationscited by 266
  20. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizerscited by 235
  21. 2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 181
  22. 2022QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantizationcited by 180
  23. 2019Improving Neural Network Quantization without Retraining using Outlier Channel Splittingcited by 151
  24. 2023Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scalingcited by 61
  25. 2023Training Transformers with 4-bit Integerscited by 60

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025HadaNorm: Diffusion Transformer Quantization through Mean-Centered Transformations
  2. 2024BiDM: Pushing the Limit of Quantization for Diffusion Models
  3. 2024PQD: Post-training Quantization for Efficient Diffusion Models
  4. 2025SQ-DM: Accelerating Diffusion Models with Aggressive Quantization and Temporal Sparsity
  5. 2025MPQ-DMv2: Flexible Residual Mixed Precision Quantization for Low-Bit Diffusion Models with Temporal Distillation
  6. 2025Diffusion Model Quantization: A Review
  7. 2024Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers
  8. 2024Efficiency Meets Fidelity: A Novel Quantization Framework for Stable Diffusion