mlsanityData to 14 Oct 2026

Unified Stochastic Framework for Neural Network Quantization and Pruning

arXiv 2412.18184 ↗First posted 24 Dec 2024cs.LG cs.NA math.NAcited by 3

Not on the map. Why?

Abstract (arXiv)

Quantization and pruning are two essential techniques for compressing neural networks, yet they are often treated independently, with limited theoretical analysis connecting them. This paper introduces a unified framework for post-training quantization and pruning using stochastic path-following algorithms. Our approach builds on the Stochastic Path Following Quantization (SPFQ) method, extending its applicability to pruning and low-bit quantization, including challenging 1-bit regimes. By incorporating a scaling parameter and generalizing the stochastic operator, the proposed method achieves robust error correction and yields rigorous theoretical error bounds for both quantization and pruning as well as their combination.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 27 of its 36 references that are in the map. First posted on arXiv 24 Dec 2024.

Builds on

27 of 36 references.

References · 27

  1. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  2. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  3. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 812
  4. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 650
  5. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  6. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 592
  7. 2019Rigging the Lottery: Making All Tickets Winnerscited by 495
  8. 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 479
  9. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 471
  10. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
  11. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 443
  12. 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 385
15 more
  1. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 274
  2. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 269
  3. 2019MetaPruning: Meta Learning for Automatic Neural Network Channel Pruningcited by 227
  4. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 217
  5. 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 193
  6. 2017Data-Driven Sparse Structure Selection for Deep Neural Networkscited by 181
  7. 2019Improving Neural Network Quantization without Retraining using Outlier Channel Splittingcited by 151
  8. 2021Group Fisher Pruning for Practical Network Compressioncited by 131
  9. 2020Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programmingcited by 92
  10. 2020Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpotcited by 66
  11. 2022Post-training Quantization for Neural Networks with Provable Guaranteescited by 33
  12. 2020Discrepancy Minimization via a Self-Balancing Walkcited by 22
  13. 2020A Greedy Algorithm for Quantizing Neural Networkscited by 22
  14. 2022A simple approach for quantizing neural networkscited by 8
  15. 2024Accumulator-Aware Post-Training Quantization for Large Language Modelscited by 6

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025Provable Post-Training Quantization: Theoretical Analysis of OPTQ and Qronoscited by 16
  2. 2025AWP: Activation-Aware Weight Pruning and Quantization with Projected Gradient Descentcited by 3
  3. 2025Theoretical Compression Bounds for Wide Multilayer Perceptronscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2023SPFQ: A Stochastic Algorithm and Its Error Analysis for Neural Network Quantization
  2. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Survey
  3. 2025Automatic Joint Structured Pruning and Quantization for Efficient Neural Network Training and Compression
  4. 2025Integrating Pruning with Quantization for Efficient Deep Neural Networks Compression
  5. 2021Ps and Qs: Quantization-aware pruning for efficient low latency neural network inference
  6. 2023Towards Optimal Compression: Joint Pruning and Quantization
  7. 2026Inner Product Aware Quantization: Provably Fast, Accurate, and Adaptive Algorithms
  8. 2023Pruning vs Quantization: Which is Better?