mlsanityData to 7 Oct 2026

TQ-DiT: Efficient Time-Aware Quantization for Diffusion Transformers

arXiv 2502.04056 ↗First posted 6 Feb 2025cs.LG eess.SPcited by 2

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion transformers (DiTs) combine transformer architectures with diffusion models. However, their computational complexity imposes significant limitations on real-time applications and sustainability of AI systems. In this study, we aim to enhance the computational efficiency through model quantization, which represents the weights and activation values with lower precision. Multi-region quantization (MRQ) is introduced to address the asymmetric distribution of network values in DiT blocks by allocating two scaling parameters to sub-regions. Additionally, time-grouping quantization (TGQ) is proposed to reduce quantization error caused by temporal variation in activations. The experimental results show that the proposed algorithm achieves performance comparable to the original full-precision model with only a 0.29 increase in FID at W8A8. Furthermore, it outperforms other baselines at W6A6, thereby confirming its suitability for low-bit quantization. These results highlight the potential of our method to enable efficient real-time generative models.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 25 of its 33 references that are in the map. First posted on arXiv 6 Feb 2025.

Builds on

25 of 33 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024PTQ4DiT: Post-training Quantization for Diffusion Transformers25 May
  2. 2023PTQD: Accurate Post-Training Quantization for Diffusion Models18 May
  3. 2023MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizer25 Mar
  4. 2023Q-Diffusion: Quantizing Diffusion Models8 Feb
  5. 2022Scalable Diffusion Models with Transformers19 Dec
  6. 2022Diffusion Models: A Comprehensive Survey of Methods and Applications2 Sep
  7. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  8. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  9. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 16

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2016Improved Techniques for Training GANscited by 2,938
  5. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  6. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  7. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
  8. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  9. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  10. 2021A White Paper on Neural Network Quantizationcited by 453
  11. 2021Post-Training Quantization for Vision Transformercited by 274
  12. 2021Generating Images with Sparse Representationscited by 262
4 more
  1. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  2. 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 185
  3. 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 173
  4. 2022Generative Joint Source-Channel Coding for Semantic Image Transmissioncited by 55

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026HiLo-Token: Input-Adaptive High-Low Frequency Token Compression for Efficient Image Editingcited by 0
  2. 2026Vitality-Aware Compression for Efficient Image-to-Shape Diffusion Transformerscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024TaQ-DiT: Time-aware Quantization for Diffusion Transformers
  2. 2025Post-Training Quantization for Diffusion Transformer via Hierarchical Timestep Grouping
  3. 2024DiTAS: Quantizing Diffusion Transformers via Enhanced Activation Smoothing
  4. 2024HQ-DiT: Efficient Diffusion Transformer with FP4 Hybrid Quantization
  5. 2025RobuQ: Pushing DiTs to W1.58A2 via Robust Activation Quantization
  6. 2025TreeQ: Pushing the Quantization Boundary of Diffusion Transformer via Tree-Structured Mixed-Precision Search
  7. 2024Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers
  8. 2026AdaTSQ: Pushing the Pareto Frontier of Diffusion Transformers via Temporal-Sensitivity Quantization