mlsanityData to 7 Oct 2026

PTQ4DiT: Post-training Quantization for Diffusion Transformers

arXiv 2405.16005 ↗First posted 25 May 2024cs.CVcited by 77

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

The recent introduction of Diffusion Transformers (DiTs) has demonstrated exceptional capabilities in image generation by using a different backbone architecture, departing from traditional U-Nets and embracing the scalable nature of transformers. Despite their advanced capabilities, the wide deployment of DiTs, particularly for real-time applications, is currently hampered by considerable computational demands at the inference stage. Post-training Quantization (PTQ) has emerged as a fast and data-efficient solution that can significantly reduce computation and memory footprint by using low-bit weights and activations. However, its applicability to DiTs has not yet been explored and faces non-trivial difficulties due to the unique design of DiTs. In this paper, we propose PTQ4DiT, a specifically designed PTQ method for DiTs. We discover two primary quantization challenges inherent in DiTs, notably the presence of salient channels with extreme magnitudes and the temporal variability in distributions of salient activation over multiple timesteps. To tackle these challenges, we propose Channel-wise Salience Balancing (CSB) and Spearmen's ρ-guided Salience Calibration (SSC). CSB leverages the complementarity property of channel magnitudes to redistribute the extremes, alleviating quantization errors for both activations and weights. SSC extends this approach by dynamically adjusting the balanced salience to capture the temporal variations in activation. Additionally, to eliminate extra computational costs caused by PTQ4DiT during inference, we design an offline re-parameterization strategy for DiTs. Experiments demonstrate that our PTQ4DiT successfully quantizes DiTs to 8-bit precision (W8A8) while preserving comparable generation ability and further enables effective quantization to 4-bit weight precision (W4A8) for the first time.

Key facts. Cited by 77 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 54 of its 65 references that are in the map. First posted on arXiv 25 May 2024.

Builds on

54 of 65 references.

Closest ancestors · 14

Same direction, newest first

  1. 2024QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuning6 Feb
  2. 2023Towards Accurate Post-training Quantization for Diffusion Models30 May
  3. 2023PTQD: Accurate Post-Training Quantization for Diffusion Models18 May
  4. 2023MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizer25 Mar
  5. 2023Q-Diffusion: Quantizing Diffusion Models8 Feb
  6. 2022Scalable Diffusion Models with Transformers19 Dec
  7. 2022Post-training Quantization on Diffusion Models28 Nov
  8. 2022Diffusion Models in Vision: A Survey10 Sep
  9. 2022Diffusion Models: A Comprehensive Survey of Methods and Applications2 Sep
  10. 2022Your ViT is Secretly a Hybrid Discriminative-Generative Diffusion Model16 Aug
  11. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  12. 2021Diffusion Models Beat GANs on Image Synthesis11 May
2 more
  1. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  2. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  7. 2020End-to-End Object Detection with Transformerscited by 7,384
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2016Layer Normalizationcited by 5,579
  10. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  11. 2016Improved Techniques for Training GANscited by 2,938
  12. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
28 more
  1. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  2. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  3. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  4. 2019Learned Step Size Quantizationcited by 604
  5. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  6. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
  7. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  8. 2021A White Paper on Neural Network Quantizationcited by 453
  9. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 403
  10. 2021Generating Images with Sparse Representationscited by 262
  11. 2018A Note on the Inception Scorecited by 225
  12. 2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 176
  13. 2022QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantizationcited by 175
  14. 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 173
  15. 2023ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Modelscited by 156
  16. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 134
  17. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  18. 2023PB-LLM: Partially Binarized Large Language Modelscited by 80
  19. 2022PD-Quant: Post-Training Quantization based on Prediction Difference Metriccited by 73
  20. 2023QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Modelscited by 63
  21. 2022NoisyQuant: Noisy Bias-Enhanced Post-Training Activation Quantization for Vision Transformerscited by 59
  22. 2023Oscillation-free Quantization for Low-bit Vision Transformerscited by 55
  23. 2024IntactKV: Improving Large Language Model Quantization by Keeping Pivot Tokens Intactcited by 52
  24. 2024MoPE-CLIP: Structured Pruning for Efficient Vision-Language Models with Module-wise Pruning Error Metriccited by 30
  25. 2024Token Transformation Matters: Towards Faithful Post-hoc Explanation for Vision Transformercited by 19
  26. 2024On the Faithfulness of Vision Transformer Explanationscited by 19
  27. 2023Jumping through Local Minima: Quantization in the Loss Landscape of Vision Transformerscited by 18
  28. 2024COMQ: A Backpropagation-Free Algorithm for Post-Training Quantizationcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 36; 2026: 31.

77 papers.

Sort:

By area:

  1. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 165
  2. 2024SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Modelscited by 161
  3. 2024ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generationcited by 110
  4. 2024Q-DiT: Accurate Post-Training Quantization for Diffusion Transformerscited by 91
  5. 2025Sparse-vDiT: Unleashing the Power of Sparse Attention to Accelerate Video Diffusion Transformerscited by 34
  6. 20241.58-bit FLUXcited by 31
  7. 2024MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantizationcited by 26
  8. 2026QuantVLA: Scale-Calibrated Post-Training Quantization for Vision-Language-Action Modelscited by 25
  9. 2025Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMscited by 21
  10. 2025QVGen: Pushing the Limit of Quantized Video Generative Modelscited by 20
  11. 2025Q-VDiT: Towards Accurate Quantization and Distillation of Video-Generation Diffusion Transformerscited by 20
  12. 2025FPSAttention: Training-Aware FP8 and Sparsity Co-Design for Fast Video Diffusioncited by 19
38 more
  1. 2025TR-DQ: Time-Rotation Diffusion Quantizationcited by 18
  2. 2025Quantized Visual Geometry Grounded Transformercited by 16
  3. 2025Accelerating Diffusion Transformer via Gradient-Optimized Cachecited by 15
  4. 2025DVD-Quant: Data-free Video Diffusion Transformers Quantizationcited by 14
  5. 2024DiTAS: Quantizing Diffusion Transformers via Enhanced Activation Smoothingcited by 13
  6. 2025CacheQuant: Comprehensively Accelerated Diffusion Modelscited by 13
  7. 2025MPQ-DMv2: Flexible Residual Mixed Precision Quantization for Low-Bit Diffusion Models with Temporal Distillationcited by 12
  8. 2025S2Q-VDiT: Accurate Quantized Video Diffusion Transformer with Salient Data and Sparse Token Distillationcited by 12
  9. 2025LRQ-DiT: Log-Rotation Post-Training Quantization of Diffusion Transformers for Image and Video Generationcited by 10
  10. 2026SVG-EAR: Parameter-Free Linear Compensation for Sparse Video Generation via Error-aware Routingcited by 9
  11. 2025MixDiT: Accelerating Image Diffusion Transformer Inference with Mixed-Precision MX Quantizationcited by 7
  12. 2025Diffusion Model Quantization: A Reviewcited by 6
  13. 2025DMQ: Dissecting Outliers of Diffusion Models for Post-Training Quantizationcited by 6
  14. 2026Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generationcited by 6
  15. 2024Temporal Feature Matters: A Framework for Diffusion Model Quantizationcited by 5
  16. 2025OuroMamba: A Data-Free Quantization Framework for Vision Mambacited by 5
  17. 2025Error Propagation Mechanisms and Compensation Strategies for Quantized Diffusioncited by 5
  18. 2025QuantSparse: Comprehensively Compressing Video Diffusion Transformer with Model Quantization and Attention Sparsificationcited by 5
  19. 2026Denoising as Path Planning: Training-Free Acceleration of Diffusion Models with DPCachecited by 5
  20. 2026OrbitQuant: Data-Agnostic Quantization for Image and Video Diffusion Transformerscited by 5
  21. 2025DLFR-VAE: Dynamic Latent Frame Rate VAE for Video Generationcited by 4
  22. 2025Modulated Diffusion: Accelerating Generative Modeling with Modulated Quantizationcited by 4
  23. 2025Test-Time Iterative Error Correction for Efficient Diffusion Modelscited by 4
  24. 2026AdaTSQ: Pushing the Pareto Frontier of Diffusion Transformers via Temporal-Sensitivity Quantizationcited by 4
  25. 2025Understanding Flatness in Generative Models: Its Role and Benefitscited by 3
  26. 2025AccuQuant: Simulating Multiple Denoising Steps for Quantizing Diffusion Modelscited by 3
  27. 2025TreeQ: Pushing the Quantization Boundary of Diffusion Transformer via Tree-Structured Mixed-Precision Searchcited by 3
  28. 2026PTQ4ARVG: Post-Training Quantization for AutoRegressive Visual Generation Modelscited by 3
  29. 2026Q-Drift: Quantization-Aware Drift Correction for Diffusion Model Samplingcited by 3
  30. 2026DiRotQ: Rotation-Aware Quantization for 4-bit Diffusion Transformerscited by 3
  31. 2024TaQ-DiT: Time-aware Quantization for Diffusion Transformerscited by 2
  32. 2025TQ-DiT: Efficient Time-Aware Quantization for Diffusion Transformerscited by 2
  33. 2025Post-Training Quantization for Diffusion Transformer via Hierarchical Timestep Groupingcited by 2
  34. 2025Post-Training Quantization for Audio Diffusion Transformerscited by 2
  35. 2025FraQAT: Quantization Aware Training with Fractional bitscited by 2
  36. 2026Token Pruning for In-Context Generation in Diffusion Transformerscited by 2
  37. 2026LoRaQ: Optimized Low Rank Approximation for 4-bit Quantizationcited by 2
  38. 2024DilateQuant: Accurate and Efficient Diffusion Quantization via Weight Dilationcited by 1

50 most cited of 77 All 77 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024An Analysis on Quantizing Diffusion Transformers
  2. 2024VQ4DiT: Efficient Post-Training Vector Quantization for Diffusion Transformers
  3. 2025RobuQ: Pushing DiTs to W1.58A2 via Robust Activation Quantization
  4. 2024HQ-DiT: Efficient Diffusion Transformer with FP4 Hybrid Quantization
  5. 2025CLQ: Cross-Layer Guided Orthogonal-based Quantization for Diffusion Transformers
  6. 2024PQD: Post-training Quantization for Efficient Diffusion Models
  7. 2026CentriQ: Calibration-Free Quantization of Diffusion Transformers via Exact Mean Centering
  8. 2023EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models