mlsanityData to 7 Oct 2026

NIPQ: Noise proxy-based Integrated Pseudo-Quantization

arXiv 2206.00820 ↗First posted 2 Jun 2022cs.LGcited by 35

Not on the map. Why?

Abstract (arXiv)

Straight-through estimator (STE), which enables the gradient flow over the non-differentiable function via approximation, has been favored in studies related to quantization-aware training (QAT). However, STE incurs unstable convergence during QAT, resulting in notable quality degradation in low precision. Recently, pseudoquantization training has been proposed as an alternative approach to updating the learnable parameters using the pseudo-quantization noise instead of STE. In this study, we propose a novel noise proxy-based integrated pseudoquantization (NIPQ) that enables unified support of pseudoquantization for both activation and weight by integrating the idea of truncation on the pseudo-quantization framework. NIPQ updates all of the quantization parameters (e.g., bit-width and truncation boundary) as well as the network parameters via gradient descent without STE instability. According to our extensive experiments, NIPQ outperforms existing quantization algorithms in various vision and language applications by a large margin.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Builds on 40 of its 64 references that are in the map. First posted on arXiv 2 Jun 2022.

Builds on

40 of 64 references.

References · 40

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  7. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  8. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  9. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  10. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  11. 2019Searching for MobileNetV3cited by 1,984
  12. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
28 more
  1. 2017Enhanced Deep Residual Networks for Single Image Super-Resolutioncited by 1,543
  2. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  3. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  4. 2019Learned Step Size Quantizationcited by 604
  5. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  6. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
  7. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 426
  8. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 387
  9. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 250
  10. 2019Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networkscited by 234
  11. 2018Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithmcited by 233
  12. 2020LSQ+: Improving low-bit quantization through learnable offsets and better initializationcited by 180
  13. 2018Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Losscited by 157
  14. 2022Overcoming Oscillations in Quantization-Aware Trainingcited by 116
  15. 2020Stabilizing Differentiable Architecture Search via Perturbation-based Regularizationcited by 102
  16. 2020Robust Quantization: One Model to Rule Them Allcited by 63
  17. 2020PROFIT: A Novel Training Method for sub-4-bit MobileNet Modelscited by 50
  18. 2020PAMS: Quantized Super-Resolution via Parameterized Max Scalecited by 46
  19. 2020Differentiable Joint Pruning and Quantization for Hardware Efficiencycited by 45
  20. 2021Differentiable Model Compression via Pseudo Quantization Noisecited by 44
  21. 2018Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implicationscited by 35
  22. 2018Neural Cache: Bit-Serial In-Cache Acceleration of Deep Neural Networkscited by 29
  23. 2021Smart at what cost? Characterising Mobile Deep Neural Networks in the wildcited by 25
  24. 2021Sharpness-aware Quantization for Deep Neural Networkscited by 25
  25. 2018NICE: Noise Injection and Clamping Estimation for Neural Network Quantizationcited by 24
  26. 2022Dynamic Dual Trainable Bounds for Ultra-low Precision Super-Resolution Networkscited by 20
  27. 2021APNN-TC: Accelerating Arbitrary Precision Neural Networks on Ampere GPU Tensor Corescited by 12
  28. 2017High performance ultra-low-precision convolutions on mobile devicescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 5; 2024: 13; 2025: 13; 2026: 2.

35 papers.

Sort:

By area:

  1. 2023Temporal Dynamic Quantization for Diffusion Modelscited by 81
  2. 2024Advancing Multimodal Large Language Models with Quantization-Aware Scale Learning for Efficient Adaptationcited by 15
  3. 2024Retraining-free Model Quantization via One-Shot Weight-Coupling Learningcited by 13
  4. 2024QT-DoG: Quantization-aware Training for Domain Generalizationcited by 13
  5. 2022Quantization Robust Federated Learning for Efficient Inference on Heterogeneous Devicescited by 12
  6. 2025Low-bit Model Quantization for Deep Neural Networks: A Surveycited by 11
  7. 2024PTQ4VM: Post-Training Quantization for Visual Mambacited by 9
  8. 2024GIFT-SW: Gaussian noise Injected Fine-Tuning of Salient Weights for LLMscited by 8
  9. 2023MetaMix: Meta-state Precision Searcher for Mixed-precision Activation Quantizationcited by 7
  10. 2025Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applicationscited by 6
  11. 2024FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Modelscited by 5
  12. 2024MetaAug: Meta-Data Augmentation for Post-Training Quantizationcited by 5
23 more
  1. 2025Improving the Straight-Through Estimator with Zeroth-Order Informationcited by 5
  2. 2022QuantNAS for super resolution: searching for efficient quantization-friendly architectures against quantization noisecited by 4
  3. 2023QBitOpt: Fast and Accurate Bitwidth Reallocation during Trainingcited by 4
  4. 2025LOTION: Smoothing the Optimization Landscape for Quantized Trainingcited by 4
  5. 2025ProfilingAgent: Profiling-Guided Agentic Reasoning for Adaptive Model Optimizationcited by 3
  6. 2024HLQ: Fast and Efficient Backpropagation via Hadamard Low-rank Quantizationcited by 2
  7. 2025Stabilizing Quantization-Aware Training by Implicit-Regularization on Hessian Matrixcited by 2
  8. 2025HOT: Hadamard-based Optimized Trainingcited by 2
  9. 2025Quantization Meets OOD: Generalizable Quantization-aware Training from a Flatness Perspectivecited by 2
  10. 2025LampQ: Towards Accurate Layer-wise Mixed Precision Quantization for Vision Transformerscited by 2
  11. 2024Minimize Quantization Output Error with Bias Compensationcited by 1
  12. 2024Scheduling Weight Transitions for Quantization-Aware Trainingcited by 1
  13. 2024ARQ: A Mixed-Precision Quantization Framework for Accurate and Certifiably Robust DNNscited by 1
  14. 2025Merge-Friendly Post-Training Quantization for Multi-Target Domain Adaptationcited by 1
  15. 2025Efficiently Training A Flat Neural Network Before It has been Quantizatedcited by 1
  16. 2023Modular Quantization-Aware Training for 6D Object Pose Estimationcited by 0
  17. 2023Fully Quantized Always-on Face Detector Considering Mobile Image Sensorscited by 0
  18. 2024Constraint Guided Model Quantization of Neural Networkscited by 0
  19. 2024Behavior Backdoor for Deep Learning Modelscited by 0
  20. 2025Precision Neural Network Quantization via Learnable Adaptive Modulescited by 0
  21. 2025BD-Net: Has Depth-Wise Convolution Ever Been Applied in Binary Neural Networks?cited by 0
  22. 2026Efficient Tuning Before Low-Bit Post-Training Quantization for Stochastic Gradient Descent-optimized Modelscited by 0
  23. 2026Quantization-Aware Pre-Training with Constrained Empirical Weight Distributioncited by 0