mlsanityData to 14 Oct 2026

PQS (Prune, Quantize, and Sort): Low-Bitwidth Accumulation of Dot Products in Neural Network Computations

arXiv 2504.09064 ↗First posted 12 Apr 2025cs.LG cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

We present PQS, which uses three techniques together - Prune, Quantize, and Sort - to achieve low-bitwidth accumulation of dot products in neural network computations. In conventional quantized (e.g., 8-bit) dot products, partial results are accumulated into wide (e.g., 32-bit) accumulators to avoid overflows when accumulating intermediate partial sums. However, such wide accumulators increase memory bandwidth usage and reduce energy efficiency. We show that iterative N:M pruning in floating point followed by quantization to 8 (or fewer) bits, and accumulation of partial products in a sorted order ("small to large") allows for accurate, compressed models with short dot product lengths that do not require wide accumulators. We design, analyze, and implement the PQS algorithm to eliminate accumulation overflows at inference time for several neural networks. Our method offers a 2.5x reduction in accumulator bitwidth while achieving model accuracy on par with floating-point baselines for multiple image classification tasks.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 12 of its 38 references that are in the map. First posted on arXiv 12 Apr 2025.

Builds on

12 of 38 references.

References · 12

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  3. 2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 527
  4. 2020MCUNet: Tiny Deep Learning on IoT Devicescited by 273
  5. 2021Learning N:M Fine-grained Structured Sparse Neural Networks From Scratchcited by 179
  6. 2020Differentiable Joint Pruning and Quantization for Hardware Efficiencycited by 46
  7. 2021Ps and Qs: Quantization-aware pruning for efficient low latency neural network inferencecited by 31
  8. 2023A2Q: Accumulator-Aware Quantization with Guaranteed Overflow Avoidancecited by 12
  9. 2020Quantization of Deep Neural Networks for Accumulator-constrained Processorscited by 9
  10. 2024A2Q+: Improving Accumulator-Aware Weight Quantizationcited by 5
  11. 2021dCSR: A Memory-Efficient Sparse Matrix Representation for Parallel Neural Network Inferencecited by 4
  12. 2023MEMA Runtime Framework: Minimizing External Memory Accesses for TinyML on Microcontrollerscited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025The Cambrian Explosion of Mixed-Precision Matrix Multiplication for Quantized Deep Learning Inferencecited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2023Quantized Neural Networks for Low-Precision Accumulation with Guaranteed Overflow Avoidance
  2. 2024Towards Cheaper Inference in Deep Networks with Lower Bit-Width Accumulators
  3. 2021A White Paper on Neural Network Quantization
  4. 2026FastKron: Efficient Quantization with Kronecker-Factored Hessians
  5. 2021A Survey of Quantization Methods for Efficient Neural Network Inference
  6. 2025Quantization vs Pruning: Insights from the Strong Lottery Ticket Hypothesis
  7. 2025Theoretical Compression Bounds for Wide Multilayer Perceptrons
  8. 2021PQK: Model Compression via Pruning, Quantization, and Knowledge Distillation