mlsanityData to 14 Oct 2026

PARQ: Piecewise-Affine Regularized Quantization

arXiv 2503.15748 ↗First posted 19 Mar 2025cs.LG math.OCcited by 5

Not on the map. Why?

Abstract (arXiv)

We develop a principled method for quantization-aware training (QAT) of large-scale machine learning models. Specifically, we show that convex, piecewise-affine regularization (PAR) can effectively induce the model parameters to cluster towards discrete values. We minimize PAR-regularized loss functions using an aggregate proximal stochastic gradient method (AProx) and prove that it has last-iterate convergence. Our approach provides an interpretation of the straight-through estimator (STE), a widely used heuristic for QAT, as the asymptotic form of PARQ. We conduct experiments to demonstrate that PARQ obtains competitive performance on convolution- and transformer-based vision tasks.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 36 of its 46 references that are in the map. First posted on arXiv 19 Mar 2025.

Builds on

36 of 46 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  3. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  4. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  5. 2021Convex optimizationcited by 2,515
  6. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  7. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  8. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  9. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  10. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  11. 2019Learned Step Size Quantizationcited by 628
  12. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
24 more
  1. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 592
  2. 2014Convex Optimization: Algorithms and Complexitycited by 575
  3. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
  4. 2017Efficient Processing of Deep Neural Networks: A Tutorial and Surveycited by 434
  5. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 426
  6. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  7. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 269
  8. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 251
  9. 2019Understanding Straight-Through Estimator in Training Activation Quantized Neural Netscited by 231
  10. 2016Trained Ternary Quantizationcited by 208
  11. 2020BinaryBERT: Pushing the Limit of BERT Quantizationcited by 178
  12. 2019Forward and Backward Information Retention for Accurate Binary Neural Networkscited by 169
  13. 2020Training with Quantization Noise for Extreme Model Compressioncited by 145
  14. 2020Training Binary Neural Networks with Real-to-Binary Convolutionscited by 112
  15. 2019MultiGrain: a unified image embedding for classes and instancescited by 85
  16. 2017Training Quantized Nets: A Deeper Understandingcited by 75
  17. 2018ProxQuant: Quantized Neural Networks via Proximal Operatorscited by 59
  18. 2018BinaryRelax: A Relaxation Approach For Training Deep Neural Networks With Quantized Weightscited by 24
  19. 2020Least squares binary quantization of neural networkscited by 21
  20. 2019Mirror Descent View for Neural Network Quantizationcited by 16
  21. 2021Demystifying and Generalizing BinaryConnectcited by 11
  22. 2016Quantization and Training of Low Bit-Width Convolutional Neural Networks for Object Detectioncited by 8
  23. 2014Generalized Conditional Gradient for Sparse Estimationcited by 6
  24. 2024Understanding Neural Network Binarization with Forward and Backward Proximal Quantizerscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025High-Dimensional Learning Dynamics of Quantized Models with Straight-Through Estimatorcited by 4
  2. 2025Beyond Discreteness: Sample Complexity Analysis of Straight-Through Estimator for 1-bit Quantizationcited by 2
  3. 2026Understanding Quantization-Aware Training: Gradients at Quantized Weights Bias to the Low-Loss Basincited by 1
  4. 2025Quantization through Piecewise-Affine Regularization: Optimization and Statistical Guaranteescited by 0
  5. 2026SURGE: Surrogate Gradient Adaptation in Binary Neural Networkscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Cat: Post-Training Quantization Error Reduction via Cluster-based Affine Transformation
  2. 2024Instance-Aware Group Quantization for Vision Transformers
  3. 2025FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximation
  4. 2024MPTQ-ViT: Mixed-Precision Post-Training Quantization for Vision Transformer
  5. 2026StableQAT: Stable Quantization-Aware Training at Ultra-Low Bitwidths
  6. 2024AffineQuant: Affine Transformation Quantization for Large Language Models
  7. 2021Regularized Classification-Aware Quantization
  8. 2024Art and Science of Quantizing Large-Scale Models: A Comprehensive Overview