mlsanityData to 7 Oct 2026

ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation

arXiv 2406.02540 ↗First posted 4 Jun 2024cs.CVcited by 110

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion transformers have demonstrated remarkable performance in visual generation tasks, such as generating realistic images or videos based on textual instructions. However, larger model sizes and multi-frame processing for video generation lead to increased computational and memory costs, posing challenges for practical deployment on edge devices. Post-Training Quantization (PTQ) is an effective method for reducing memory costs and computational complexity. When quantizing diffusion transformers, we find that existing quantization methods face challenges when applied to text-to-image and video tasks. To address these challenges, we begin by systematically analyzing the source of quantization error and conclude with the unique challenges posed by DiT quantization. Accordingly, we design an improved quantization scheme: ViDiT-Q (Video & Image Diffusion Transformer Quantization), tailored specifically for DiT models. We validate the effectiveness of ViDiT-Q across a variety of text-to-image and video models, achieving W8A8 and W4A8 with negligible degradation in visual quality and metrics. Additionally, we implement efficient GPU kernels to achieve practical 2-2.5x memory saving and a 1.4-1.7x end-to-end latency speedup.

Key facts. Cited by 110 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 65 of its 91 references that are in the map. First posted on arXiv 4 Jun 2024.

Builds on

65 of 91 references.

Closest ancestors · 24

Same direction, newest first

  1. 2024MixDQ: Memory-Efficient Few-Step Text-to-Image Diffusion Models with Metric-Decoupled Mixed Precision Quantization28 May
  2. 2024PTQ4DiT: Post-training Quantization for Diffusion Transformers25 May
  3. 2024TMPQ-DM: Joint Timestep Reduction and Quantization Precision Selection for Efficient Diffusion Models15 Apr
  4. 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation7 Mar
  5. 2024EDA-DM: Enhanced Distribution Alignment for Post-Training Quantization of Diffusion Models9 Jan
  6. 2023A Unified Sampling Framework for Solver Searching of Diffusion Probabilistic Models12 Dec
  7. 2023Efficient Quantization Strategies for Latent Diffusion Models9 Dec
  8. 2023DiffiT: Diffusion Vision Transformers for Image Generation4 Dec
  9. 2023DeepCache: Accelerating Diffusion Models for Free1 Dec
  10. 2023Post-training Quantization for Text-to-Image Diffusion Models with Progressive Calibration and Activation Relaxing10 Nov
  11. 2023Effective Quantization for Diffusion Models on CPUs2 Nov
  12. 2023EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models5 Oct
12 more
  1. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis30 Sep
  2. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  3. 2023Fast Training of Diffusion Models with Masked Transformers15 Jun
  4. 2023OMS-DPM: Optimizing the Model Schedule for Diffusion Probabilistic Models15 Jun
  5. 2023Temporal Dynamic Quantization for Diffusion Models4 Jun
  6. 2023PTQD: Accurate Post-Training Quantization for Diffusion Models18 May
  7. 2023Q-Diffusion: Quantizing Diffusion Models8 Feb
  8. 2022Scalable Diffusion Models with Transformers19 Dec
  9. 2022Post-training Quantization on Diffusion Models28 Nov
  10. 2022All are Worth Words: A ViT Backbone for Diffusion Models25 Sep
  11. 2022Classifier-Free Diffusion Guidance26 Jul
  12. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec

Other references · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  5. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  6. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  7. 2022Video Diffusion Modelscited by 2,280
  8. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  9. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  10. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  11. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  12. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
29 more
  1. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  2. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  3. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  4. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  5. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  6. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  7. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 598
  8. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
  9. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  10. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  11. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  12. 2021A White Paper on Neural Network Quantizationcited by 453
  13. 2024SpinQuant: LLM quantization with learned rotationscited by 446
  14. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 426
  15. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 397
  16. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 387
  17. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
  18. 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 314
  19. 2021Post-Training Quantization for Vision Transformercited by 274
  20. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 265
  21. 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Servingcited by 255
  22. 2021GODIVA: Generating Open-DomaIn Videos from nAtural Descriptionscited by 254
  23. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Servingcited by 200
  24. 2020ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functionscited by 183
  25. 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 173
  26. 2021FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformercited by 148
  27. 2024ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generationcited by 130
  28. 2023A Practical Mixed Precision Algorithm for Post-Training Quantizationcited by 23
  29. 2023Quantization Variation: A New Perspective on Training Transformers with Low-Bit Precisioncited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 47; 2026: 50.

110 papers.

Sort:

By area:

  1. 2024Deep Compression Autoencoder for Efficient High-Resolution Diffusion Modelscited by 209
  2. 2025Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsitycited by 204
  3. 2024SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Modelscited by 161
  4. 2024SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Accelerationcited by 158
  5. 2025Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Modelscited by 126
  6. 2024FasterCache: Training-Free Video Diffusion Model Acceleration with High Qualitycited by 115
  7. 2024SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantizationcited by 113
  8. 2025SpargeAttention: Accurate and Training-free Sparse Attention Accelerating Any Model Inferencecited by 113
  9. 2024DiTFastAttn: Attention Compression for Diffusion Transformer Modelscited by 109
  10. 2025Radial Attention: O(nlog n) Sparse Attention with Energy Decay for Long Video Generationcited by 64
  11. 20241.58-bit FLUXcited by 31
  12. 2026QuantVLA: Scale-Calibrated Post-Training Quantization for Vision-Language-Action Modelscited by 25
38 more
  1. 2026LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generationcited by 25
  2. 2025HiFlow: Training-free High-Resolution Image Generation with Flow-Aligned Guidancecited by 21
  3. 2025SageAttention2++: A More Efficient Implementation of SageAttention2cited by 21
  4. 2025Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMscited by 21
  5. 2024MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Modelscited by 20
  6. 2025QuantCache: Adaptive Importance-Guided Quantization with Hierarchical Latent and Layer Caching for Video Generationcited by 20
  7. 2025QVGen: Pushing the Limit of Quantized Video Generative Modelscited by 20
  8. 2025Q-VDiT: Towards Accurate Quantization and Distillation of Video-Generation Diffusion Transformerscited by 20
  9. 2025FPSAttention: Training-Aware FP8 and Sparsity Co-Design for Fast Video Diffusioncited by 19
  10. 2026Flow caching for autoregressive video generationcited by 19
  11. 2025TR-DQ: Time-Rotation Diffusion Quantizationcited by 18
  12. 2025PAROAttention: Pattern-Aware ReOrdering for Efficient Sparse and Quantized Attention in Visual Generation Modelscited by 17
  13. 2024LiteVAR: Compressing Visual Autoregressive Modelling with Efficient Attention and Quantizationcited by 16
  14. 2025Attend to Not Attended: Structure-then-Detail Token Merging for Post-training DiT Accelerationcited by 16
  15. 2025Quantized Visual Geometry Grounded Transformercited by 16
  16. 2025DC-VideoGen: Efficient Video Generation with Deep Compression Video Autoencodercited by 15
  17. 2025On-device Sora: Enabling Training-Free Diffusion-based Text-to-Video Generation for Mobile Devicescited by 14
  18. 2025DVD-Quant: Data-free Video Diffusion Transformers Quantizationcited by 14
  19. 2025S2Q-VDiT: Accurate Quantized Video Diffusion Transformer with Salient Data and Sparse Token Distillationcited by 12
  20. 2025Low-bit Model Quantization for Deep Neural Networks: A Surveycited by 11
  21. 2025LRQ-DiT: Log-Rotation Post-Training Quantization of Diffusion Transformers for Image and Video Generationcited by 10
  22. 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 10
  23. 2025LinVideo: A Post-Training Framework towards O(n) Attention in Efficient Video Generationcited by 9
  24. 2026Sparse Forcing: Native Trainable Sparse Attention for Real-time Autoregressive Diffusion Video Generationcited by 9
  25. 2024ENAT: Rethinking Spatial-temporal Interactions in Token-based Image Synthesiscited by 8
  26. 2025MixDiT: Accelerating Image Diffusion Transformer Inference with Mixed-Precision MX Quantizationcited by 7
  27. 2025DC-Gen: Post-Training Diffusion Acceleration with Deeply Compressed Latent Spacecited by 7
  28. 2025Massive Activations are the Key to Local Detail Synthesis in Diffusion Transformerscited by 7
  29. 2026SALAD: Achieve High-Sparsity Attention via Efficient Linear Attention Tuning for Video Diffusion Transformercited by 7
  30. 2026Fine-Grained Post-Training Quantization for Large Vision Language Models with Quantization-Aware Integrated Gradientscited by 7
  31. 2024Qua2SeDiMo: Quantifiable Quantization Sensitivity of Diffusion Modelscited by 6
  32. 2025FP4DiT: Towards Effective Floating Point Quantization for Diffusion Transformerscited by 6
  33. 2025Diffusion Model Quantization: A Reviewcited by 6
  34. 2025Foresight: Adaptive Layer Reuse for Accelerated and High-Quality Text-to-Video Generationcited by 6
  35. 2025DMQ: Dissecting Outliers of Diffusion Models for Post-Training Quantizationcited by 6
  36. 2026SenCache: Accelerating Diffusion Model Inference via Sensitivity-Aware Cachingcited by 6
  37. 2026Efficient Video Diffusion Models: Advancements and Challengescited by 6
  38. 2026Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generationcited by 6

50 most cited of 110 All 110 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers
  2. 2024VQ4DiT: Efficient Post-Training Vector Quantization for Diffusion Transformers
  3. 2024TaQ-DiT: Time-aware Quantization for Diffusion Transformers
  4. 2025TreeQ: Pushing the Quantization Boundary of Diffusion Transformer via Tree-Structured Mixed-Precision Search
  5. 2024TerDiT: Ternary Diffusion Models with Transformers
  6. 2025Post-Training Quantization for Diffusion Transformer via Hierarchical Timestep Grouping
  7. 2024QVD: Post-training Quantization for Video Diffusion Models
  8. 2024DiTAS: Quantizing Diffusion Transformers via Enhanced Activation Smoothing