# Any Orthogonal Transform Will Do: Range Reduction Explains Rotation-Based Quantization for Super-Resolution

arXiv: https://arxiv.org/abs/2601.21069 · HTML: https://mlsanity.com/paper/2601.21069/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 55 references that are in the map.
- First posted on arXiv 28 Jan 2026.

## Builds on (most cited first, 30 of 31)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs](https://mlsanity.com/paper/2404.00456/) (2024)
- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [SpinQuant: LLM quantization with learned rotations](https://mlsanity.com/paper/2405.16406/) (2024)
- [PACT: Parameterized Clipping Activation for Quantized Neural Networks](https://mlsanity.com/paper/1805.06085/) (2018)
- [QuIP: 2-Bit Quantization of Large Language Models With Guarantees](https://mlsanity.com/paper/2307.13304/) (2023)
- [QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks](https://mlsanity.com/paper/2402.04396/) (2024)
- [Sketch-based Manga Retrieval using Manga109 Dataset](https://mlsanity.com/paper/1510.04389/) (2015)
- [LSQ+: Improving low-bit quantization through learnable offsets and better initialization](https://mlsanity.com/paper/2004.09576/) (2020)
- [PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantization](https://mlsanity.com/paper/2111.12293/) (2021)
- [RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformers](https://mlsanity.com/paper/2212.08254/) (2022)
- [FlatQuant: Flatness Matters for LLM Quantization](https://mlsanity.com/paper/2410.09426/) (2024)
- [MambaIRv2: Attentive State Space Restoration](https://mlsanity.com/paper/2411.15269/) (2024)
- [Towards Accurate Post-Training Quantization for Vision Transformer](https://mlsanity.com/paper/2303.14341/) (2023)
- [NoisyQuant: Noisy Bias-Enhanced Post-Training Activation Quantization for Vision Transformers](https://mlsanity.com/paper/2211.16056/) (2022)
- [PAMS: Quantized Super-Resolution via Parameterized Max Scale](https://mlsanity.com/paper/2011.04212/) (2020)
- [DAQ: Channel-Wise Distribution-Aware Quantization for Deep Image Super-Resolution Networks](https://mlsanity.com/paper/2012.11230/) (2020)
- [CADyQ: Content-Aware Dynamic Quantization for Image Super-Resolution](https://mlsanity.com/paper/2207.10345/) (2022)
- [Dynamic Dual Trainable Bounds for Ultra-low Precision Super-Resolution Networks](https://mlsanity.com/paper/2203.03844/) (2022)
- [Fully Quantized Image Super-Resolution Networks](https://mlsanity.com/paper/2011.14265/) (2020)
- [CABM: Content-Aware Bit Mapping for Single Image Super-Resolution Network with Large Input](https://mlsanity.com/paper/2304.06454/) (2023)
- [2DQuant: Low-bit Post-Training Quantization for Image Super-Resolution](https://mlsanity.com/paper/2406.06649/) (2024)
- [AdaBM: On-the-Fly Adaptive Bit Mapping for Image Super-Resolution](https://mlsanity.com/paper/2404.03296/) (2024)
- [HadaNorm: Diffusion Transformer Quantization through Mean-Centered Transformations](https://mlsanity.com/paper/2506.09932/) (2025)
- [Overcoming Distribution Mismatch in Quantizing Image Super-Resolution Networks](https://mlsanity.com/paper/2307.13337/) (2023)

## Abstract (arXiv)

Rotation-based quantization -- multiplying weights and activations by a Hadamard matrix before quantizing -- is now standard, but why it works is explained informally, via incoherence, the central limit theorem, or kurtosis, and rarely tested directly. We ask which property of the post-rotation distribution drives the gain, using paired hypothesis tests on the weights and activations of an image super-resolution transformer. Three candidate properties separate cleanly. Range reduction replicates under every orthogonal transform we tried -- Sylvester Hadamard, QuIP#-Paley, DCT-II, and random orthogonal matrices -- and transfers to a state-space backbone and to CNN weights. Improved normality also replicates. The increased mass near zero, by contrast, is significant only under the zero-padded Sylvester construction and disappears under unpadded transforms, so we attribute it to the padding, not the transform. The end task follows the same pattern: over a grid of three scale factors, three bitwidths and five benchmarks, rotation improves PSNR in all 45 configurations against an otherwise identical un-rotated quantizer at the same nominal bitwidth (the rotated arm carries 0.267 bits/parameter of Sylvester padding at 4 bits and stays ahead in every cell when charged for it), while substituting DCT-II or a random orthogonal matrix costs only 0.06-0.07 dB, less than the padding the Hadamard is charged for. The choice of kernel is therefore an implementation decision: the Hadamard's advantage is its +/-1 entries and O(n log n) butterflies. We package the recipe as CompSRT, which improves on 2DQuant in 44 of 45 configurations at equal nominal bitwidth -- charged for its padding that edge holds at 3 and 2 bits but not at 4. CondiQuant remains ahead at every operating point by 0.03-1.40 dB, which we report.