mlsanityData to 7 Oct 2026

Q&C: When Quantization Meets Cache in Efficient Image Generation

arXiv 2503.02508 ↗First posted 4 Mar 2025cs.CV eess.IVcited by 0

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Quantization and cache mechanisms are typically applied individually for efficient Diffusion Transformers (DiTs), each demonstrating notable potential for acceleration. However, the promoting effect of combining the two mechanisms on efficient generation remains under-explored. Through empirical investigation, we find that the combination of quantization and cache mechanisms for DiT is not straightforward, and two key challenges lead to severe catastrophic performance degradation: (i) the sample efficacy of calibration datasets in post-training quantization (PTQ) is significantly eliminated by cache operation; (ii) the combination of the above mechanisms introduces more severe exposure bias within sampling distribution, resulting in amplified error accumulation in the image generation process. In this work, we take advantage of these two acceleration mechanisms and propose a hybrid acceleration method by tackling the above challenges, aiming to further improve the efficiency of DiTs while maintaining excellent generation capability. Concretely, a temporal-aware parallel clustering (TAP) is designed to dynamically improve the sample selection efficacy for the calibration within PTQ for different diffusion steps. A variance compensation (VC) strategy is derived to correct the sampling distribution. It mitigates exposure bias through an adaptive correction factor generation. Extensive experiments have shown that our method has accelerated DiTs by 12.7x while preserving competitive generation capability. The code will be available at https://github.com/xinding-sys/Quant-Cache.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 50 of its 62 references that are in the map. First posted on arXiv 4 Mar 2025.

Builds on

50 of 62 references.

Closest ancestors · 23

Same direction, newest first

  1. 2024FORA: Fast-Forward Caching in Diffusion Transformer Acceleration1 Jul
  2. 2024ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation4 Jun
  3. 2024Learning-to-Cache: Accelerating Diffusion Transformer via Layer Caching3 Jun
  4. 2024Delta-DiT: A Training-Free Acceleration Method Tailored for Diffusion Transformers3 Jun
  5. 2024HQ-DiT: Efficient Diffusion Transformer with FP4 Hybrid Quantization30 May
  6. 2024MixDQ: Memory-Efficient Few-Step Text-to-Image Diffusion Models with Metric-Decoupled Mixed Precision Quantization28 May
  7. 2024PTQ4DiT: Post-training Quantization for Diffusion Transformers25 May
  8. 2024TerDiT: Ternary Diffusion Models with Transformers23 May
  9. 2024EDA-DM: Enhanced Distribution Alignment for Post-Training Quantization of Diffusion Models9 Jan
  10. 2023FRDiff : Feature Reuse for Universal Training-free Acceleration of Diffusion Models6 Dec
  11. 2023Cache Me if You Can: Accelerating Diffusion Models through Block Caching6 Dec
  12. 2023EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models5 Oct
11 more
  1. 2023On Error Propagation of Diffusion Models9 Aug
  2. 2023Alleviating Exposure Bias in Diffusion Models through Sampling with Shifted Time Steps24 May
  3. 2023Q-Diffusion: Quantizing Diffusion Models8 Feb
  4. 2023Input Perturbation Reduces Exposure Bias in Diffusion Models27 Jan
  5. 2022Scalable Diffusion Models with Transformers19 Dec
  6. 2022Post-training Quantization on Diffusion Models28 Nov
  7. 2022Diffusion Models in Vision: A Survey10 Sep
  8. 2022Diffusion Models: A Comprehensive Survey of Methods and Applications2 Sep
  9. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  10. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  11. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 27

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2020End-to-End Object Detection with Transformerscited by 7,384
  3. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  4. 2016Improved Techniques for Training GANscited by 2,938
  5. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  6. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  7. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
  8. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  9. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
  10. 2016Self-critical Sequence Training for Image Captioningcited by 544
  11. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  12. 2021A White Paper on Neural Network Quantizationcited by 453
15 more
  1. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 328
  2. 2021Generating Images with Sparse Representationscited by 262
  3. 2018A Note on the Inception Scorecited by 225
  4. 2023QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Modelscited by 154
  5. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 134
  6. 2019Generalization in Generation: A closer look at Exposure Biascited by 123
  7. 2023Compressing LLMs: The Truth is Rarely Pure and Never Simplecited by 59
  8. 2017DeepCache: Principled Cache for Mobile Deep Visioncited by 57
  9. 2023Oscillation-free Quantization for Low-bit Vision Transformerscited by 55
  10. 2023On the Impact of Calibration Data in Post-training Quantization and Pruningcited by 51
  11. 2019Fighting Quantization Bias With Biascited by 30
  12. 2023Enhancing Computation Efficiency in Large Language Models through Weight and Activation Quantizationcited by 20
  13. 2023ZeroQuant(4+2): Redefining LLMs Quantization with a New FP6-Centric Strategy for Diverse Generative Taskscited by 14
  14. 2018Faster Matrix Completion Using Randomized SVDcited by 5
  15. 2017Fast Approximate Spectral Clustering for Dynamic Networkscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026AdaCorrection: Adaptive Offset Cache Correction for Accurate Diffusion Transformers
  2. 2025Accelerating Diffusion Transformer via Error-Optimized Cache
  3. 2025Accelerating Diffusion Transformer via Increment-Calibrated Caching with Channel-Aware Singular Value Decomposition
  4. 2026AdaTSQ: Pushing the Pareto Frontier of Diffusion Transformers via Temporal-Sensitivity Quantization
  5. 2025TreeQ: Pushing the Quantization Boundary of Diffusion Transformer via Tree-Structured Mixed-Precision Search
  6. 2026Frequency-Aware Error-Bounded Caching for Accelerating Diffusion Transformers
  7. 2025SQ-DM: Accelerating Diffusion Models with Aggressive Quantization and Temporal Sparsity
  8. 2024TaQ-DiT: Time-aware Quantization for Diffusion Transformers