# A2Q+: Improving Accumulator-Aware Weight Quantization

arXiv: https://arxiv.org/abs/2401.10432 · HTML: https://mlsanity.com/paper/2401.10432/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 53 references that are in the map.
- First posted on arXiv 19 Jan 2024.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks](https://mlsanity.com/paper/1602.07868/) (2016)
- [Data-Free Quantization Through Weight Equalization and Bias Correction](https://mlsanity.com/paper/1906.04721/) (2019)
- [Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation](https://mlsanity.com/paper/2004.09602/) (2020)
- [Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networks](https://mlsanity.com/paper/1909.13144/) (2019)
- [Sparse GPU Kernels for Deep Learning](https://mlsanity.com/paper/2006.10901/) (2020)
- [Trained Quantization Thresholds for Accurate and Efficient Fixed-Point Inference of Deep Neural Networks](https://mlsanity.com/paper/1903.08066/) (2019)
- [Fast Sparse ConvNets](https://mlsanity.com/paper/1911.09723/) (2019)
- [A Mean Field Theory of Batch Normalization](https://mlsanity.com/paper/1902.08129/) (2019)
- [FINN-R: An End-to-End Deep-Learning Framework for Fast Exploration of Quantized Neural Networks](https://mlsanity.com/paper/1809.04570/) (2018)
- [Rigid-Motion Scattering for Texture Classification](https://mlsanity.com/paper/1403.1687/) (2014)
- [SHE: A Fast and Accurate Deep Neural Network for Encrypted Data](https://mlsanity.com/paper/1906.00148/) (2019)
- [Ps and Qs: Quantization-aware pruning for efficient low latency neural network inference](https://mlsanity.com/paper/2102.11289/) (2021)
- [Understanding INT4 Quantization for Transformer Models: Latency Speedup, Composability, and Failure Cases](https://mlsanity.com/paper/2301.12017/) (2023)
- [Streamlined Deployment for Quantized Neural Networks](https://mlsanity.com/paper/1709.04060/) (2017)
- [A2Q: Accumulator-Aware Quantization with Guaranteed Overflow Avoidance](https://mlsanity.com/paper/2308.13504/) (2023)
- [Deep Neural Networks for Encrypted Inference with TFHE](https://mlsanity.com/paper/2302.10906/) (2023)
- [Quantization of Deep Neural Networks for Accumulator-constrained Processors](https://mlsanity.com/paper/2004.11783/) (2020)
- [TensorQuant - A Simulation Toolbox for Deep Neural Network Quantization](https://mlsanity.com/paper/1710.05758/) (2017)
- [A Competitive Edge: Can FPGAs Beat GPUs at DCNN Inference Acceleration in Resource-Limited Edge Computing Applications?](https://mlsanity.com/paper/2102.00294/) (2021)
- [An Energy-Efficient Edge Computing Paradigm for Convolution-based Image Upsampling](https://mlsanity.com/paper/2107.07647/) (2021)
- [Shedding the Bits: Pushing the Boundaries of Quantization with Minifloats on FPGAs](https://mlsanity.com/paper/2311.12359/) (2023)

## Built on by (most cited first, 5 of 5)

- [Pushing the Limits of Block Rotations in Post-Training Quantization](https://mlsanity.com/paper/2601.22347/) (2026)
- [HGQ: High Granularity Quantization for Real-time Neural Networks on FPGAs](https://mlsanity.com/paper/2405.00645/) (2024)
- [Accumulator-Aware Post-Training Quantization for Large Language Models](https://mlsanity.com/paper/2409.17092/) (2024)
- [PQS (Prune, Quantize, and Sort): Low-Bitwidth Accumulation of Dot Products in Neural Network Computations](https://mlsanity.com/paper/2504.09064/) (2025)
- [Practical and Private Hybrid ML Inference with Fully Homomorphic Encryption](https://mlsanity.com/paper/2509.01253/) (2025)

## Abstract (arXiv)

Quantization techniques commonly reduce the inference costs of neural networks by restricting the precision of weights and activations. Recent studies show that also reducing the precision of the accumulator can further improve hardware efficiency at the risk of numerical overflow, which introduces arithmetic errors that can degrade model accuracy. To avoid numerical overflow while maintaining accuracy, recent work proposed accumulator-aware quantization (A2Q), a quantization-aware training method that constrains model weights during training to safely use a target accumulator bit width during inference. Although this shows promise, we demonstrate that A2Q relies on an overly restrictive constraint and a sub-optimal weight initialization strategy that each introduce superfluous quantization error. To address these shortcomings, we introduce: (1) an improved bound that alleviates accumulator constraints without compromising overflow avoidance; and (2) a new strategy for initializing quantized weights from pre-trained floating-point checkpoints. We combine these contributions with weight normalization to introduce A2Q+. We support our analysis with experiments that show A2Q+ significantly improves the trade-off between accumulator bit width and model accuracy and characterize new trade-offs that arise as a consequence of accumulator constraints.