mlsanityData to 14 Oct 2026

KroQuant: Kronecker-Structured Block Transforms for Efficient Post-Training Quantization of Diffusion Transformers

arXiv 2607.21446 ↗First posted 23 Jul 2026cs.LG cs.CVcited by 0

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Post-training quantization (PTQ) of diffusion transformers (DiTs) to W4A4 severely degrades output quality, because activations entering each linear layer contain outliers that 4-bit formats cannot represent. The standard fix applies an invertible linear transform to the activations and its inverse to the weights before quantizing both. Normalization layers between blocks force this transform to run online at every denoising step, making its inference computation cost the binding design constraint. Existing options trade quantization quality for inference cost: per-channel scaling (SmoothQuant) is computationally cheap but impacts the magnitude of the channels, which can harm quantization accuracy; fixed Hadamard transforms yield better quantization accuracy but require large block sizes that incur a high online cost; learned full-d invertible transforms calibrate best but entail a prohibitive dense d × d matrix multiplication (GEMM) per layer per step. We propose KroQuant, a PTQ method that applies a learned Kronecker-structured invertible transform to each 32-element block of the activation, storing less than half the parameters of per-channel scaling. The block-local structure runs as small tensor-core GEMMs, and on an MI350 GPU the KroQuant quantizer kernel is up to 14% faster than the SmoothQuant kernel. Offline LoRaQ weight calibration then absorbs the residual per-weight quantization error. On PixArt-Sigma, SANA, and FLUX.1-schnell at W4A4 (MXFP4e2), KroQuant produces outputs closer to the FP reference than SVDQuant and LoRaQ on MJHQ-30K and SDCI, while preserving or improving image quality.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 14 of its 22 references that are in the map. First posted on arXiv 23 Jul 2026.

Builds on

14 of 22 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026LoRaQ: Optimized Low Rank Approximation for 4-bit Quantization20 Apr
  2. 2024Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers25 Jun
  3. 2024ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation4 Jun
  4. 2024MixDQ: Memory-Efficient Few-Step Text-to-Image Diffusion Models with Metric-Decoupled Mixed Precision Quantization28 May
  5. 2024PTQ4DiT: Post-training Quantization for Diffusion Transformers25 May
  6. 2022Post-training Quantization on Diffusion Models28 Nov

Other references · 8

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  3. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  4. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  5. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  6. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  7. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  8. 2023A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captionscited by 108

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024An Analysis on Quantizing Diffusion Transformers
  2. 2026Deflating the Hessian: Rank-4 W4A4 Quantization for Multimodal Diffusion Transformers
  3. 2024TaQ-DiT: Time-aware Quantization for Diffusion Transformers
  4. 2025Q-Sched: Pushing the Boundaries of Few-Step Diffusion Models with Quantization-Aware Scheduling
  5. 2025LRQ-DiT: Log-Rotation Post-Training Quantization of Diffusion Transformers for Image and Video Generation
  6. 2025Post-Training Quantization for Diffusion Transformer via Hierarchical Timestep Grouping
  7. 2026DiRotQ: Rotation-Aware Quantization for 4-bit Diffusion Transformers
  8. 2025HadaNorm: Diffusion Transformer Quantization through Mean-Centered Transformations