# PowerQuant: Automorphism Search for Non-Uniform Quantization

arXiv: https://arxiv.org/abs/2301.09858 · HTML: https://mlsanity.com/paper/2301.09858/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 59 references that are in the map.
- First posted on arXiv 24 Jan 2023.

## Builds on (most cited first, 30 of 41)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [Importance Estimation for Neural Network Pruning](https://mlsanity.com/paper/1906.10771/) (2019)
- [Quantizing deep convolutional networks for efficient inference: A whitepaper](https://mlsanity.com/paper/1806.08342/) (2018)
- [Data-Free Quantization Through Weight Equalization and Bias Correction](https://mlsanity.com/paper/1906.04721/) (2019)
- [LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks](https://mlsanity.com/paper/1807.10029/) (2018)
- [ZeroQ: A Novel Zero Shot Quantization Framework](https://mlsanity.com/paper/2001.00281/) (2020)
- [I-BERT: Integer-only BERT Quantization](https://mlsanity.com/paper/2101.01321/) (2021)
- [Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights](https://mlsanity.com/paper/1702.03044/) (2017)
- [Quantized Convolutional Neural Networks for Mobile Devices](https://mlsanity.com/paper/1512.06473/) (2015)
- [Compressing Deep Convolutional Networks using Vector Quantization](https://mlsanity.com/paper/1412.6115/) (2014)
- [Low-bit Quantization of Neural Networks for Efficient Inference](https://mlsanity.com/paper/1902.06822/) (2019)
- [LSQ+: Improving low-bit quantization through learnable offsets and better initialization](https://mlsanity.com/paper/2004.09576/) (2020)
- [Improving Neural Network Quantization without Retraining using Outlier Channel Splitting](https://mlsanity.com/paper/1901.09504/) (2019)
- [Compressing Neural Networks with the Hashing Trick](https://mlsanity.com/paper/1504.04788/) (2015)
- [Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networks](https://mlsanity.com/paper/1909.13144/) (2019)
- [Convolutional Neural Networks using Logarithmic Data Representation](https://mlsanity.com/paper/1603.01025/) (2016)
- [Generative Low-bitwidth Data Free Quantization](https://mlsanity.com/paper/2003.03603/) (2020)

## Built on by (most cited first, 20 of 20)

- [I&S-ViT: An Inclusive & Stable Method for Pushing the Limit of Post-Training ViTs Quantization](https://mlsanity.com/paper/2311.10126/) (2023)
- [Communication-Efficient Split Learning via Adaptive Feature-Wise Compression](https://mlsanity.com/paper/2307.10805/) (2023)
- [A Comprehensive Survey of Compression Algorithms for Language Models](https://mlsanity.com/paper/2401.15347/) (2024)
- [Low-bit Model Quantization for Deep Neural Networks: A Survey](https://mlsanity.com/paper/2505.05530/) (2025)
- [NUPES : Non-Uniform Post-Training Quantization via Power Exponent Search](https://mlsanity.com/paper/2308.05600/) (2023)
- [SEAM: Searching Transferable Mixed-Precision Quantization Policy through Large Margin Regularization](https://mlsanity.com/paper/2302.06845/) (2023)
- [SL-ACC: A Communication-Efficient Split Learning Framework with Adaptive Channel-wise Compression](https://mlsanity.com/paper/2508.12984/) (2025)
- [Efficient and Robust Quantization-aware Training via Adaptive Coreset Selection](https://mlsanity.com/paper/2306.07215/) (2023)
- [Communication Efficient Split Learning of ViTs with Attention-based Double Compression](https://mlsanity.com/paper/2509.15058/) (2025)
- [Fighting over-fitting with quantization for learning deep neural networks on noisy labels](https://mlsanity.com/paper/2303.11803/) (2023)
- [SAfER: Layer-Level Sensitivity Assessment for Efficient and Robust Neural Network Inference](https://mlsanity.com/paper/2308.04753/) (2023)
- [Network Memory Footprint Compression Through Jointly Learnable Codebooks and Mappings](https://mlsanity.com/paper/2309.17361/) (2023)
- [Ex Uno Pluria: Insights on Ensembling in Low Precision Number Systems](https://mlsanity.com/paper/2411.14860/) (2024)
- [Exploiting Information Redundancy in Attention Maps for Extreme Quantization of Vision Transformers](https://mlsanity.com/paper/2508.16311/) (2025)
- [SL-FAC: A Communication-Efficient Split Learning Framework with Frequency-Aware Compression](https://mlsanity.com/paper/2604.07316/) (2026)
- [PoTAcc: A Pipeline for End-to-End Acceleration of Power-of-Two Quantized DNNs](https://mlsanity.com/paper/2605.06082/) (2026)
- [Designing strong baselines for ternary neural network quantization through support and mass equalization](https://mlsanity.com/paper/2306.17442/) (2023)
- [Gradient-Based Post-Training Quantization: Challenging the Status Quo](https://mlsanity.com/paper/2308.07662/) (2023)
- [Quantization Range Estimation for Convolutional Neural Networks](https://mlsanity.com/paper/2510.04044/) (2025)
- [NANQ: Noise-Floor-Aware Mixed-Precision Non-Uniform Quantization for Analog Compute-in-Memory](https://mlsanity.com/paper/2608.02700/) (2026)

## Abstract (arXiv)

Deep neural networks (DNNs) are nowadays ubiquitous in many domains such as computer vision. However, due to their high latency, the deployment of DNNs hinges on the development of compression techniques such as quantization which consists in lowering the number of bits used to encode the weights and activations. Growing concerns for privacy and security have motivated the development of data-free techniques, at the expanse of accuracy. In this paper, we identity the uniformity of the quantization operator as a limitation of existing approaches, and propose a data-free non-uniform method. More specifically, we argue that to be readily usable without dedicated hardware and implementation, non-uniform quantization shall not change the nature of the mathematical operations performed by the DNN. This leads to search among the continuous automorphisms of (ℝ_+^*,×), which boils down to the power functions defined by their exponent. To find this parameter, we propose to optimize the reconstruction error of each layer: in particular, we show that this procedure is locally convex and admits a unique solution. At inference time, we show that our approach, dubbed PowerQuant, only require simple modifications in the quantized DNN activation functions. As such, with only negligible overhead, it significantly outperforms existing methods in a variety of configurations.