Any Orthogonal Transform Will Do: Range Reduction Explains Rotation-Based Quantization for Super-Resolution
Not on the map. Why?
Abstract (arXiv)
Rotation-based quantization -- multiplying weights and activations by a Hadamard matrix before quantizing -- is now standard, but why it works is explained informally, via incoherence, the central limit theorem, or kurtosis, and rarely tested directly. We ask which property of the post-rotation distribution drives the gain, using paired hypothesis tests on the weights and activations of an image super-resolution transformer. Three candidate properties separate cleanly. Range reduction replicates under every orthogonal transform we tried -- Sylvester Hadamard, QuIP#-Paley, DCT-II, and random orthogonal matrices -- and transfers to a state-space backbone and to CNN weights. Improved normality also replicates. The increased mass near zero, by contrast, is significant only under the zero-padded Sylvester construction and disappears under unpadded transforms, so we attribute it to the padding, not the transform. The end task follows the same pattern: over a grid of three scale factors, three bitwidths and five benchmarks, rotation improves PSNR in all 45 configurations against an otherwise identical un-rotated quantizer at the same nominal bitwidth (the rotated arm carries 0.267 bits/parameter of Sylvester padding at 4 bits and stays ahead in every cell when charged for it), while substituting DCT-II or a random orthogonal matrix costs only 0.06-0.07 dB, less than the padding the Hadamard is charged for. The choice of kernel is therefore an implementation decision: the Hadamard's advantage is its +/-1 entry and O(n log n) butterflies. We package the recipe as CompSRT, which improves on 2DQuant in 44 of 45 configurations at equal nominal bitwidth -- charged for its padding that edge holds at 3 and 2 bits but not at 4. CondiQuant remains ahead at every operating point by 0.03-1.40 dB, which we report.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 55 references that are in the map. First posted on arXiv 28 Jan 2026.
Builds on
31 of 55 references.
References · 31
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,908
- 2021SwinIR: Image Restoration Using Swin Transformercited by 1,887
- 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
- 2017Enhanced Deep Residual Networks for Single Image Super-Resolutioncited by 1,552
- 2018Residual Dense Network for Image Super-Resolutioncited by 799
- 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
- 2019Learned Step Size Quantizationcited by 628
- 2024SpinQuant: LLM quantization with learned rotationscited by 471
- 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
- 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
- 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
19 more
- 2015Sketch-based Manga Retrieval using Manga109 Datasetcited by 329
- 2020LSQ+: Improving low-bit quantization through learnable offsets and better initializationcited by 189
- 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 179
- 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 137
- 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 131
- 2024MambaIRv2: Attentive State Space Restorationcited by 94
- 2023Towards Accurate Post-Training Quantization for Vision Transformercited by 71
- 2022NoisyQuant: Noisy Bias-Enhanced Post-Training Activation Quantization for Vision Transformerscited by 61
- 2020PAMS: Quantized Super-Resolution via Parameterized Max Scalecited by 48
- 2020DAQ: Channel-Wise Distribution-Aware Quantization for Deep Image Super-Resolution Networkscited by 31
- 2022CADyQ: Content-Aware Dynamic Quantization for Image Super-Resolutioncited by 27
- 2022Dynamic Dual Trainable Bounds for Ultra-low Precision Super-Resolution Networkscited by 21
- 2020Fully Quantized Image Super-Resolution Networkscited by 19
- 2023CABM: Content-Aware Bit Mapping for Single Image Super-Resolution Network with Large Inputcited by 16
- 20242DQuant: Low-bit Post-Training Quantization for Image Super-Resolutioncited by 14
- 2024AdaBM: On-the-Fly Adaptive Bit Mapping for Image Super-Resolutioncited by 9
- 2025HadaNorm: Diffusion Transformer Quantization through Mean-Centered Transformationscited by 7
- 2023Overcoming Distribution Mismatch in Quantizing Image Super-Resolution Networkscited by 5
- 2025CondiQuant: Condition Number Based Low-Bit Quantization for Image Super-Resolutioncited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026QuantSR+: Pushing the Limit of Quantized Image Super-Resolution Networks
- 2026Q-DiT4SR: Exploration of Detail-Preserving Diffusion Transformer Quantization for Real-World Image Super-Resolution
- 2026OrbitQuant: Data-Agnostic Quantization for Image and Video Diffusion Transformers
- 2026RPFQ-ViT: Rotated Phase-Frame Quantization for Extremely Low-Bit Weights in Vision Transformers
- 2025Outlier-Aware Post-Training Quantization for Image Super-Resolution
- 2025QArtSR: Quantization via Reverse-Module and Timestep-Retraining in One-Step Diffusion based Image Super-Resolution
- 2023Exploring Frequency-Inspired Optimization in Transformer for Efficient Single Image Super-Resolution
- 2026Zero-Shot Quantization via Weight-Space Arithmetic