mlsanityData to 7 Oct 2026

Temporal Feature Matters: A Framework for Diffusion Model Quantization

arXiv 2407.19547 ↗First posted 28 Jul 2024cs.CVcited by 5

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

The Diffusion models, widely used for image generation, face significant challenges related to their broad applicability due to prolonged inference times and high memory demands. Efficient Post-Training Quantization (PTQ) is crucial to address these issues. However, unlike traditional models, diffusion models critically rely on the time-step for the multi-round denoising. Typically, each time-step is encoded into a hypersensitive temporal feature by several modules. Despite this, existing PTQ methods do not optimize these modules individually. Instead, they employ unsuitable reconstruction objectives and complex calibration methods, leading to significant disturbances in the temporal feature and denoising trajectory, as well as reduced compression efficiency. To address these challenges, we introduce a novel quantization framework that includes three strategies: 1) TIB-based Maintenance: Based on our innovative Temporal Information Block (TIB) definition, Temporal Information-aware Reconstruction (TIAR) and Finite Set Calibration (FSC) are developed to efficiently align original temporal features. 2) Cache-based Maintenance: Instead of indirect and complex optimization for the related modules, pre-computing and caching quantized counterparts of temporal features are developed to minimize errors. 3) Disturbance-aware Selection: Employ temporal feature errors to guide a fine-grained selection between the two maintenance strategies for further disturbance reduction. This framework preserves most of the temporal information and ensures high-quality end-to-end generation. Extensive testing on various datasets, diffusion models and hardware confirms our superior performance and acceleration.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 85 of its 111 references that are in the map. First posted on arXiv 28 Jul 2024.

Builds on

85 of 111 references.

Closest ancestors · 42

Same direction, newest first

  1. 2024Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers25 Jun
  2. 2024BitsFusion: 1.99 bits Weight Quantization of Diffusion Model6 Jun
  3. 2024ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation4 Jun
  4. 2024PTQ4DiT: Post-training Quantization for Diffusion Transformers25 May
  5. 2024QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuning6 Feb
  6. 2023Cache Me if You Can: Accelerating Diffusion Models through Block Caching6 Dec
  7. 2023DeepCache: Accelerating Diffusion Models for Free1 Dec
  8. 2023Adversarial Diffusion Distillation28 Nov
  9. 2023TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models27 Nov
  10. 2023EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models5 Oct
  11. 2023Softmax Bias Correction for Quantized Generative Models4 Sep
  12. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
30 more
  1. 2023Temporal Dynamic Quantization for Diffusion Models4 Jun
  2. 2023PTQD: Accurate Post-Training Quantization for Diffusion Models18 May
  3. 2023Speed Is All You Need: On-Device Acceleration of Large Diffusion Models via GPU-Aware Optimizations21 Apr
  4. 2023Q-Diffusion: Quantizing Diffusion Models8 Feb
  5. 2022Scalable Diffusion Models with Transformers19 Dec
  6. 2022Post-training Quantization on Diffusion Models28 Nov
  7. 2022Diffusion Probabilistic Model Made Slim27 Nov
  8. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models2 Nov
  9. 2022Denoising MCMC for Accelerating Diffusion-Based Generative Models29 Sep
  10. 2022Classifier-Free Diffusion Guidance26 Jul
  11. 2022Estimating the Optimal Covariance with Imperfect Mean in Diffusion Probabilistic Models15 Jun
  12. 2022gDDIM: Generalized denoising diffusion implicit models11 Jun
  13. 2022How Much is Enough? A Study on Diffusion Times in Score-based Generative Models10 Jun
  14. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  15. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  16. 2022Accelerating Diffusion Models via Early Stop of the Diffusion Process25 May
  17. 2022Fast Sampling of Diffusion Models with Exponential Integrator29 Apr
  18. 2022Pseudo Numerical Methods for Diffusion Models on Manifolds20 Feb
  19. 2022Truncated Diffusion Probabilistic Models and Diffusion-based Adversarial Auto-Encoders19 Feb
  20. 2022Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Quality11 Feb
  21. 2022Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models17 Jan
  22. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  23. 2021Variational Diffusion Models1 Jul
  24. 2021On Fast Sampling of Diffusion Probabilistic Models31 May
  25. 2021Gotta Go Fast When Generating Data with Score-Based Models28 May
  26. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  27. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed7 Jan
  28. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  29. 2020Denoising Diffusion Implicit Models6 Oct
  30. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 43

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  7. 2022Generative Adversarial Networkscited by 5,603
  8. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  9. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  10. 2016Improved Techniques for Training GANscited by 2,938
  11. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  12. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
31 more
  1. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  2. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  3. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  4. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  5. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  6. 2019Learned Step Size Quantizationcited by 604
  7. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  8. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  9. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  10. 2021A White Paper on Neural Network Quantizationcited by 453
  11. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
  12. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 317
  13. 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 314
  14. 2021Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contractioncited by 306
  15. 2019Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networkscited by 234
  16. 2023SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Secondscited by 227
  17. 2023Cross-Entropy Loss Functions: Theoretical Analysis and Applicationscited by 206
  18. 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 185
  19. 2020Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluationcited by 183
  20. 2020LSQ+: Improving low-bit quantization through learnable offsets and better initializationcited by 180
  21. 2022QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantizationcited by 175
  22. 2021FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformercited by 148
  23. 2021Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillationcited by 131
  24. 2023SwiftBrush: One-Step Text-to-Image Diffusion Model with Variational Score Distillationcited by 109
  25. 2022On the Optimization Landscape of Neural Collapse under MSE Loss: Global Optimality with Unconstrained Featurescited by 97
  26. 2020Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programmingcited by 89
  27. 2017Towards Effective Low-bitwidth Convolutional Neural Networkscited by 76
  28. 2022PD-Quant: Post-Training Quantization based on Prediction Difference Metriccited by 73
  29. 2022BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesiscited by 64
  30. 2024LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkitcited by 32
  31. 2022Efficient Adaptive Activation Rounding for Post-Training Quantizationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025QVGen: Pushing the Limit of Quantized Video Generative Modelscited by 20
  2. 2025SlimInfer: Accelerating Long-Context LLM Inference via Dynamic Token Pruningcited by 13
  3. 2025LLMC+: Benchmarking Vision-Language Model Compression with a Plug-and-play Toolkitcited by 7
  4. 2026Focus-dLLM: Accelerating Long-Context Diffusion LLM Inference via Confidence-Guided Context Focusingcited by 6
  5. 2026Flash-VAED: Plug-and-Play VAE Decoders for Efficient Video Generationcited by 5

Similar, not cited

Close in wording, no citation either way

  1. 2026TASQ: Temporal-Adaptive Bit Sparsification Quantization for Diffusion Models
  2. 2024TCAQ-DM: Timestep-Channel Adaptive Quantization for Diffusion Models
  3. 2026TMPDiff: Temporal Mixed-Precision for Diffusion Models
  4. 2026Gradient-Aligned Calibration for Post-Training Quantization of Diffusion Models
  5. 2024Timestep-Aware Correction for Quantized Diffusion Models
  6. 2024TaQ-DiT: Time-aware Quantization for Diffusion Transformers
  7. 2024TMPQ-DM: Joint Timestep Reduction and Quantization Precision Selection for Efficient Diffusion Models
  8. 2025TR-DQ: Time-Rotation Diffusion Quantization