mlsanityData to 14 Oct 2026

Symmetry Regularization and Saturating Nonlinearity for Robust Quantization

arXiv 2208.00338 ↗First posted 31 Jul 2022cs.LG cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Robust quantization improves the tolerance of networks for various implementations, allowing reliable output in different bit-widths or fragmented low-precision arithmetic. In this work, we perform extensive analyses to identify the sources of quantization error and present three insights to robustify a network against quantization: reduction of error propagation, range clamping for error minimization, and inherited robustness against quantization. Based on these insights, we propose two novel methods called symmetry regularization (SymReg) and saturating nonlinearity (SatNL). Applying the proposed methods during training can enhance the robustness of arbitrary neural networks against quantization on existing post-training quantization (PTQ) and quantization-aware training (QAT) algorithms and enables us to obtain a single weight flexible enough to maintain the output quality under various conditions. We conduct extensive studies on CIFAR and ImageNet datasets and validate the effectiveness of the proposed methods.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 33 of its 46 references that are in the map. First posted on arXiv 31 Jul 2022.

Builds on

33 of 46 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  4. 2014Going Deeper with Convolutionscited by 4,241
  5. 2019Searching for MobileNetV3cited by 1,994
  6. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  7. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
  8. 2019Learned Step Size Quantizationcited by 628
  9. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 569
  10. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  11. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
  12. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 402
21 more
  1. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 336
  2. 2021ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networkscited by 293
  3. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 269
  4. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 261
  5. 2020Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluationcited by 188
  6. 2022QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantizationcited by 180
  7. 2018Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Losscited by 160
  8. 2019Improving Neural Network Quantization without Retraining using Outlier Channel Splittingcited by 151
  9. 2019Robust Learning with Jacobian Regularizationcited by 129
  10. 2017Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networkscited by 109
  11. 2019Loss Aware Post-training Quantizationcited by 97
  12. 2019Defensive Quantization: When Efficiency Meets Robustnesscited by 95
  13. 2021Characterizing signal propagation to close the performance gap in unnormalized ResNetscited by 88
  14. 2020Robust Quantization: One Model to Rule Them Allcited by 63
  15. 2020HAWQV3: Dyadic Neural Network Quantizationcited by 60
  16. 2018Value-aware Quantization for Training and Inference of Neural Networkscited by 52
  17. 2020PROFIT: A Novel Training Method for sub-4-bit MobileNet Modelscited by 50
  18. 2018Post-training 4-bit quantization of convolution networks for rapid-deploymentcited by 42
  19. 2019Fighting Quantization Bias With Biascited by 30
  20. 2018Quantization for Rapid Deployment of Deep Neural Networkscited by 20
  21. 2017High performance ultra-low-precision convolutions on mobile devicescited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2026Can Large Language Models Still Explain Themselves? Investigating the Impact of Quantization on Self-Explanationscited by 5
  2. 2025Through a Compressed Lens: Investigating The Impact of Quantization on Factual Knowledge Recallcited by 3
  3. 2025Assessing the Potential for Catastrophic Failure in Dynamic Post-Training Quantizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Robust Training of Neural Networks at Arbitrary Precision and Sparsity
  2. 2023Designing strong baselines for ternary neural network quantization through support and mass equalization
  3. 2023RobustMQ: Benchmarking Robustness of Quantized Models
  4. 2022Quantization-aware Interval Bound Propagation for Training Certifiably Robust Quantized Neural Networks
  5. 2023Benchmarking the Robustness of Quantized Models
  6. 2022ODG-Q: Robust Quantization via Online Domain Generalization
  7. 2022CSQ: Growing Mixed-Precision Quantization Scheme with Bi-level Continuous Sparsification
  8. 2024Outlier-Aware Training for Low-Bit Quantization of Structural Re-Parameterized Networks