# PQS (Prune, Quantize, and Sort): Low-Bitwidth Accumulation of Dot Products in Neural Network Computations

arXiv: https://arxiv.org/abs/2504.09064 · HTML: https://mlsanity.com/paper/2504.09064/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 38 references that are in the map.
- First posted on arXiv 12 Apr 2025.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks](https://mlsanity.com/paper/2102.00554/) (2021)
- [MCUNet: Tiny Deep Learning on IoT Devices](https://mlsanity.com/paper/2007.10319/) (2020)
- [Learning N:M Fine-grained Structured Sparse Neural Networks From Scratch](https://mlsanity.com/paper/2102.04010/) (2021)
- [Differentiable Joint Pruning and Quantization for Hardware Efficiency](https://mlsanity.com/paper/2007.10463/) (2020)
- [Ps and Qs: Quantization-aware pruning for efficient low latency neural network inference](https://mlsanity.com/paper/2102.11289/) (2021)
- [A2Q: Accumulator-Aware Quantization with Guaranteed Overflow Avoidance](https://mlsanity.com/paper/2308.13504/) (2023)
- [Quantization of Deep Neural Networks for Accumulator-constrained Processors](https://mlsanity.com/paper/2004.11783/) (2020)
- [A2Q+: Improving Accumulator-Aware Weight Quantization](https://mlsanity.com/paper/2401.10432/) (2024)
- [dCSR: A Memory-Efficient Sparse Matrix Representation for Parallel Neural Network Inference](https://mlsanity.com/paper/2111.12345/) (2021)
- [MEMA Runtime Framework: Minimizing External Memory Accesses for TinyML on Microcontrollers](https://mlsanity.com/paper/2304.05544/) (2023)

## Built on by (most cited first, 1 of 1)

- [The Cambrian Explosion of Mixed-Precision Matrix Multiplication for Quantized Deep Learning Inference](https://mlsanity.com/paper/2506.11728/) (2025)

## Abstract (arXiv)

We present PQS, which uses three techniques together - Prune, Quantize, and Sort - to achieve low-bitwidth accumulation of dot products in neural network computations. In conventional quantized (e.g., 8-bit) dot products, partial results are accumulated into wide (e.g., 32-bit) accumulators to avoid overflows when accumulating intermediate partial sums. However, such wide accumulators increase memory bandwidth usage and reduce energy efficiency. We show that iterative N:M pruning in floating point followed by quantization to 8 (or fewer) bits, and accumulation of partial products in a sorted order ("small to large") allows for accurate, compressed models with short dot product lengths that do not require wide accumulators. We design, analyze, and implement the PQS algorithm to eliminate accumulation overflows at inference time for several neural networks. Our method offers a 2.5x reduction in accumulator bitwidth while achieving model accuracy on par with floating-point baselines for multiple image classification tasks.