# Frame Quantization of Neural Networks

arXiv: https://arxiv.org/abs/2404.08131 · HTML: https://mlsanity.com/paper/2404.08131/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 44 references that are in the map.
- First posted on arXiv 11 Apr 2024.

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)
- [ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers](https://mlsanity.com/paper/2206.01861/) (2022)
- [QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantization](https://mlsanity.com/paper/2203.05740/) (2022)
- [Loss Aware Post-training Quantization](https://mlsanity.com/paper/1911.07190/) (2019)
- [SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximation](https://mlsanity.com/paper/2202.07471/) (2022)
- [Post-training Quantization for Neural Networks with Provable Guarantees](https://mlsanity.com/paper/2201.11113/) (2022)
- [BinaryRelax: A Relaxation Approach For Training Deep Neural Networks With Quantized Weights](https://mlsanity.com/paper/1801.06313/) (2018)
- [A Greedy Algorithm for Quantizing Neural Networks](https://mlsanity.com/paper/2010.15979/) (2020)
- [Adaptive Precision Training (AdaPT): A dynamic fixed point quantized training approach for DNNs](https://mlsanity.com/paper/2107.13490/) (2021)
- [A simple approach for quantizing neural networks](https://mlsanity.com/paper/2209.03487/) (2022)
- [Approximation of functions with one-bit neural networks](https://mlsanity.com/paper/2112.09181/) (2021)
- [Recurrence of Optimum for Training Weight and Activation Quantized Networks](https://mlsanity.com/paper/2012.05529/) (2020)
- [Sigma-Delta and Distributed Noise-Shaping Quantization Methods for Random Fourier Features](https://mlsanity.com/paper/2106.02614/) (2021)

## Built on by (most cited first, 2 of 2)

- [MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantization](https://mlsanity.com/paper/2406.00800/) (2024)
- [Fine-Tuning of Transformer models with Frames](https://mlsanity.com/paper/2608.26430/) (2026)

## Abstract (arXiv)

We present a post-training quantization algorithm with error estimates relying on ideas originating from frame theory. Specifically, we use first-order Sigma-Delta (SigmaDelta) quantization for finite unit-norm tight frames to quantize weight matrices and biases in a neural network. In our scenario, we derive an error bound between the original neural network and the quantized neural network in terms of step size and the number of frame elements. We also demonstrate how to leverage the redundancy of frames to achieve a quantized neural network with higher accuracy.