# NICE: Noise Injection and Clamping Estimation for Neural Network Quantization

arXiv: https://arxiv.org/abs/1810.00162 · HTML: https://mlsanity.com/paper/1810.00162/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Sep 2018.

## Built on by (most cited first, 24 of 24)

- [Mixed-Precision Neural Network Quantization via Learned Layer-wise Importance](https://mlsanity.com/paper/2203.08368/) (2022)
- [Differentiable Model Compression via Pseudo Quantization Noise](https://mlsanity.com/paper/2104.09987/) (2021)
- [NIPQ: Noise proxy-based Integrated Pseudo-Quantization](https://mlsanity.com/paper/2206.00820/) (2022)
- [Pareto-Optimal Quantized ResNet Is Mostly 4-bit](https://mlsanity.com/paper/2105.03536/) (2021)
- [Differentiable Dynamic Quantization with Mixed Precision and Adaptive Resolution](https://mlsanity.com/paper/2106.02295/) (2021)
- [Dynamic Precision Analog Computing for Neural Networks](https://mlsanity.com/paper/2102.06365/) (2021)
- [FAT: Learning Low-Bitwidth Parametric Representation via Frequency-Aware Transformation](https://mlsanity.com/paper/2102.07444/) (2021)
- [LOTION: Smoothing the Optimization Landscape for Quantized Training](https://mlsanity.com/paper/2510.08757/) (2025)
- [Low-Precision Mixed-Computation Models for Inference on Edge](https://mlsanity.com/paper/2312.02210/) (2023)
- [AMED: Automatic Mixed-Precision Quantization for Edge Devices](https://mlsanity.com/paper/2205.15437/) (2022)
- [Full-Precision and Ternarised Neural Networks with Tunnel-Diode Activation Functions: Computing and Physics Perspectives](https://mlsanity.com/paper/2503.04978/) (2025)
- [HOT: Hadamard-based Optimized Training](https://mlsanity.com/paper/2503.21261/) (2025)
- [RCT: Resource Constrained Training for Edge AI](https://mlsanity.com/paper/2103.14493/) (2021)
- [A High-Performance Adaptive Quantization Approach for Edge CNN Applications](https://mlsanity.com/paper/2107.08382/) (2021)
- [Accurate Quantization for Gait Representation Learning](https://mlsanity.com/paper/2405.13859/) (2024)
- [Merge-Friendly Post-Training Quantization for Multi-Target Domain Adaptation](https://mlsanity.com/paper/2505.23651/) (2025)
- [Starting Positions Matter: A Study on Better Weight Initialization for Neural Network Quantization](https://mlsanity.com/paper/2506.10463/) (2025)
- [Colinearity Decay: Training Quantization-Friendly ViTs with Outlier Decay](https://mlsanity.com/paper/2605.01330/) (2026)
- [Convolutional Neural Networks Quantization with Attention](https://mlsanity.com/paper/2209.15317/) (2022)
- [Error-aware Quantization through Noise Tempering](https://mlsanity.com/paper/2212.05603/) (2022)
- [RAND: Robustness Aware Norm Decay For Quantized Seq2seq Models](https://mlsanity.com/paper/2305.15536/) (2023)
- [Fully Quantized Always-on Face Detector Considering Mobile Image Sensors](https://mlsanity.com/paper/2311.01001/) (2023)
- [RepQ: Generalizing Quantization-Aware Training for Re-Parametrized Architectures](https://mlsanity.com/paper/2311.05317/) (2023)
- [SCAR-GS: Spatial Context Attention for Residuals in Progressive Gaussian Splatting](https://mlsanity.com/paper/2601.04348/) (2026)

## Abstract (arXiv)

Convolutional Neural Networks (CNN) are very popular in many fields including computer vision, speech recognition, natural language processing, to name a few. Though deep learning leads to groundbreaking performance in these domains, the networks used are very demanding computationally and are far from real-time even on a GPU, which is not power efficient and therefore does not suit low power systems such as mobile devices. To overcome this challenge, some solutions have been proposed for quantizing the weights and activations of these networks, which accelerate the runtime significantly. Yet, this acceleration comes at the cost of a larger error. The uniqname method proposed in this work trains quantized neural networks by noise injection and a learned clamping, which improve the accuracy. This leads to state-of-the-art results on various regression and classification tasks, e.g., ImageNet classification with architectures such as ResNet-18/34/50 with low as 3-bit weights and activations. We implement the proposed solution on an FPGA to demonstrate its applicability for low power real-time applications. The implementation of the paper is available at https://github.com/Lancer555/NICE