# NIPQ: Noise proxy-based Integrated Pseudo-Quantization

arXiv: https://arxiv.org/abs/2206.00820 · HTML: https://mlsanity.com/paper/2206.00820/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 64 references that are in the map.
- First posted on arXiv 2 Jun 2022.

## Builds on (most cited first, 30 of 40)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [PACT: Parameterized Clipping Activation for Quantized Neural Networks](https://mlsanity.com/paper/1805.06085/) (2018)
- [HAQ: Hardware-Aware Automated Quantization with Mixed Precision](https://mlsanity.com/paper/1811.08886/) (2018)
- [HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision](https://mlsanity.com/paper/1905.03696/) (2019)
- [HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks](https://mlsanity.com/paper/1911.03852/) (2019)
- [Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks](https://mlsanity.com/paper/1908.05033/) (2019)
- [Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithm](https://mlsanity.com/paper/1808.00278/) (2018)
- [LSQ+: Improving low-bit quantization through learnable offsets and better initialization](https://mlsanity.com/paper/2004.09576/) (2020)
- [Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Loss](https://mlsanity.com/paper/1808.05779/) (2018)
- [Overcoming Oscillations in Quantization-Aware Training](https://mlsanity.com/paper/2203.11086/) (2022)
- [Stabilizing Differentiable Architecture Search via Perturbation-based Regularization](https://mlsanity.com/paper/2002.05283/) (2020)
- [Robust Quantization: One Model to Rule Them All](https://mlsanity.com/paper/2002.07686/) (2020)
- [PROFIT: A Novel Training Method for sub-4-bit MobileNet Models](https://mlsanity.com/paper/2008.04693/) (2020)
- [PAMS: Quantized Super-Resolution via Parameterized Max Scale](https://mlsanity.com/paper/2011.04212/) (2020)

## Built on by (most cited first, 30 of 35)

- [Temporal Dynamic Quantization for Diffusion Models](https://mlsanity.com/paper/2306.02316/) (2023)
- [Advancing Multimodal Large Language Models with Quantization-Aware Scale Learning for Efficient Adaptation](https://mlsanity.com/paper/2408.03735/) (2024)
- [Retraining-free Model Quantization via One-Shot Weight-Coupling Learning](https://mlsanity.com/paper/2401.01543/) (2024)
- [QT-DoG: Quantization-aware Training for Domain Generalization](https://mlsanity.com/paper/2410.06020/) (2024)
- [Quantization Robust Federated Learning for Efficient Inference on Heterogeneous Devices](https://mlsanity.com/paper/2206.10844/) (2022)
- [Low-bit Model Quantization for Deep Neural Networks: A Survey](https://mlsanity.com/paper/2505.05530/) (2025)
- [PTQ4VM: Post-Training Quantization for Visual Mamba](https://mlsanity.com/paper/2412.20386/) (2024)
- [GIFT-SW: Gaussian noise Injected Fine-Tuning of Salient Weights for LLMs](https://mlsanity.com/paper/2408.15300/) (2024)
- [MetaMix: Meta-state Precision Searcher for Mixed-precision Activation Quantization](https://mlsanity.com/paper/2311.06798/) (2023)
- [Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications](https://mlsanity.com/paper/2508.15008/) (2025)
- [LOTION: Smoothing the Optimization Landscape for Quantized Training](https://mlsanity.com/paper/2510.08757/) (2025)
- [FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models](https://mlsanity.com/paper/2403.09904/) (2024)
- [MetaAug: Meta-Data Augmentation for Post-Training Quantization](https://mlsanity.com/paper/2407.14726/) (2024)
- [Improving the Straight-Through Estimator with Zeroth-Order Information](https://mlsanity.com/paper/2510.23926/) (2025)
- [QuantNAS for super resolution: searching for efficient quantization-friendly architectures against quantization noise](https://mlsanity.com/paper/2208.14839/) (2022)
- [QBitOpt: Fast and Accurate Bitwidth Reallocation during Training](https://mlsanity.com/paper/2307.04535/) (2023)
- [ProfilingAgent: Profiling-Guided Agentic Reasoning for Adaptive Model Optimization](https://mlsanity.com/paper/2509.05584/) (2025)
- [Scheduling Weight Transitions for Quantization-Aware Training](https://mlsanity.com/paper/2404.19248/) (2024)
- [HLQ: Fast and Efficient Backpropagation via Hadamard Low-rank Quantization](https://mlsanity.com/paper/2406.15102/) (2024)
- [Stabilizing Quantization-Aware Training by Implicit-Regularization on Hessian Matrix](https://mlsanity.com/paper/2503.11159/) (2025)
- [HOT: Hadamard-based Optimized Training](https://mlsanity.com/paper/2503.21261/) (2025)
- [Quantization Meets OOD: Generalizable Quantization-aware Training from a Flatness Perspective](https://mlsanity.com/paper/2509.00859/) (2025)
- [LampQ: Towards Accurate Layer-wise Mixed Precision Quantization for Vision Transformers](https://mlsanity.com/paper/2511.10004/) (2025)
- [Minimize Quantization Output Error with Bias Compensation](https://mlsanity.com/paper/2404.01892/) (2024)
- [Constraint Guided Model Quantization of Neural Networks](https://mlsanity.com/paper/2409.20138/) (2024)
- [ARQ: A Mixed-Precision Quantization Framework for Accurate and Certifiably Robust DNNs](https://mlsanity.com/paper/2410.24214/) (2024)
- [Precision Neural Network Quantization via Learnable Adaptive Modules](https://mlsanity.com/paper/2504.17263/) (2025)
- [Merge-Friendly Post-Training Quantization for Multi-Target Domain Adaptation](https://mlsanity.com/paper/2505.23651/) (2025)
- [Efficiently Training A Flat Neural Network Before It has been Quantizated](https://mlsanity.com/paper/2511.01462/) (2025)
- [Modular Quantization-Aware Training for 6D Object Pose Estimation](https://mlsanity.com/paper/2303.06753/) (2023)

## Abstract (arXiv)

Straight-through estimator (STE), which enables the gradient flow over the non-differentiable function via approximation, has been favored in studies related to quantization-aware training (QAT). However, STE incurs unstable convergence during QAT, resulting in notable quality degradation in low precision. Recently, pseudoquantization training has been proposed as an alternative approach to updating the learnable parameters using the pseudo-quantization noise instead of STE. In this study, we propose a novel noise proxy-based integrated pseudoquantization (NIPQ) that enables unified support of pseudoquantization for both activation and weight by integrating the idea of truncation on the pseudo-quantization framework. NIPQ updates all of the quantization parameters (e.g., bit-width and truncation boundary) as well as the network parameters via gradient descent without STE instability. According to our extensive experiments, NIPQ outperforms existing quantization algorithms in various vision and language applications by a large margin.