mlsanityData to 14 Oct 2026

DuQuant++: Fine-grained Rotation Enhances Microscaling FP4 Quantization

arXiv 2604.17789 ↗First posted 20 Apr 2026cs.CV cs.AI cs.CLcited by 8

On the map: Efficient LLM inference and serving → KV cache quantization

Abstract (arXiv)

The MXFP4 microscaling format, which partitions tensors into blocks of 32 elements sharing an E8M0 scaling factor, has emerged as a promising substrate for efficient LLM inference, backed by native hardware support on NVIDIA Blackwell Tensor Cores. However, activation outliers pose a unique challenge under this format: a single outlier inflates the shared block scale, compressing the effective dynamic range of the remaining elements and causing significant quantization error. Existing rotation-based remedies, including randomized Hadamard and learnable rotations, are data-agnostic and therefore unable to specifically target the channels where outliers concentrate. We propose DuQuant++, which adapts the outlier-aware fine-grained rotation of DuQuant to the MXFP4 format by aligning the rotation block size with the microscaling group size (B{=}32). Because each MXFP4 group possesses an independent scaling factor, the cross-block variance issue that necessitates dual rotations and a zigzag permutation in the original DuQuant becomes irrelevant, enabling DuQuant++ to replace the entire pipeline with a single outlier-aware rotation, which halves the online rotation cost while simultaneously smoothing the weight distribution. Extensive experiments on the LLaMA-3 family under MXFP4 W4A4 quantization show that DuQuant++ consistently achieves state-of-the-art performance. Our code is available at https://github.com/Hsu1023/DuQuant-v2.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in KV cache quantization, one of 12 directions in Efficient LLM inference and serving. Builds on 39 of its 47 references that are in the map. First posted on arXiv 20 Apr 2026.

Builds on

39 of 47 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving7 May
  2. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs30 Mar

Other references · 37

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  4. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  5. 2016Pointer Sentinel Mixture Modelscited by 3,096
  6. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  7. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  8. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  9. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  10. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 992
  11. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  12. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
25 more
  1. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  2. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  3. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
  4. 2024Massive Activations in Large Language Modelscited by 294
  5. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 167
  6. 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 131
  7. 2024QTIP: Quantization with Trellises and Incoherence Processingcited by 91
  8. 2024AffineQuant: Affine Transformation Quantization for Large Language Modelscited by 67
  9. 2023Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scalingcited by 61
  10. 2024IntactKV: Improving Large Language Model Quantization by Keeping Pivot Tokens Intactcited by 52
  11. 2021OMPQ: Orthogonal Mixed Precision Quantizationcited by 39
  12. 2025Training LLMs with MXFP4cited by 37
  13. 2025Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scalingcited by 35
  14. 2025Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantizationcited by 32
  15. 2026QuantVLA: Scale-Calibrated Post-Training Quantization for Vision-Language-Action Modelscited by 30
  16. 2025Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMscited by 23
  17. 2025Block Rotation is All You Need for MXFP4 Quantizationcited by 20
  18. 2024DopQ-ViT: Towards Distribution-Friendly and Outlier-Aware Post-Training Quantization for Vision Transformerscited by 19
  19. 2026ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMscited by 15
  20. 2025MicroMix: Efficient Mixed-Precision Quantization with Microscaling Formats for Large Language Modelscited by 11
  21. 2026Prune as You Generate: Online Rollout Pruning for Faster and Better RLVRcited by 10
  22. 2025Tequila: Trapping-free Ternary Quantization for Large Language Modelscited by 9
  23. 2025Scale Up Composed Image Retrieval Learning via Modification Text Generationcited by 7
  24. 2025Quaff: Quantized Parameter-Efficient Fine-Tuning under Outlier Spatial Stability Hypothesiscited by 7
  25. 2026Sherry: Hardware-Efficient 1.25-Bit Ternary Quantization via Fine-grained Sparsificationcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026Predict, Don't Iterate: Efficient Adaptive-Length Infilling for Diffusion Language Modelscited by 3
  2. 2026IV-CoT: Implicit Visual Chain-of-Thought for Structure-Aware Text-to-Image Generationcited by 2
  3. 2026FOCUS: FP4 Optimization via Coupled-Relaxation and Dual-Granularity Scalingcited by 2
  4. 2026Benchmarking Trustworthiness of SLMs: Pre-trained vs. Compressedcited by 2
  5. 2026ReSET: Accurate Latency-Critical NVFP4 Reasoning via Step-Aware Temperature Scalingcited by 0
  6. 2026SharQ: Bridging Activation Sparsity and FP4 Quantization for LLM Inferencecited by 0
  7. 2026PRQuant: Permutation Residual Quantization for Low-Overhead Inferencecited by 0
  8. 2026STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024AMXFP4: Taming Activation Outliers with Asymmetric Microscaling Floating-Point for 4-bit LLM Inference
  2. 2026Unveiling the Potential of Quantization with MXFP4: Strategies for Quantization Error Reduction
  3. 2026OCGQuant: Outlier-Companion Grouping for NVFP4 Quantization
  4. 2026BATQuant: Outlier-resilient MXFP4 Quantization via Learnable Block-wise Optimization
  5. 2026MXSens: Sensitivity-Aware Mixed-Precision Quantization for Efficient LLM Inference
  6. 2026ScaleSweep: Accurate NVFP4 Post-Training Quantization of LLMs via Block Scale Initialization
  7. 2026Quartet II: Accurate LLM Pre-Training in NVFP4 by Improved Unbiased Gradient Estimation
  8. 2024MicroScopiQ: Accelerating Foundational Models through Outlier-Aware Microscaling Quantization