# QuantNAS for super resolution: searching for efficient quantization-friendly architectures against quantization noise

arXiv: https://arxiv.org/abs/2208.14839 · HTML: https://mlsanity.com/paper/2208.14839/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 52 references that are in the map.
- First posted on arXiv 31 Aug 2022.

## Builds on (most cited first, 30 of 36)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.03443/) (2018)
- [Lightweight Image Super-Resolution with Information Multi-distillation Network](https://mlsanity.com/paper/1909.11856/) (2019)
- [ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions](https://mlsanity.com/paper/2003.03488/) (2020)
- [Deep Learning for Single Image Super-Resolution: A Brief Review](https://mlsanity.com/paper/1808.03344/) (2018)
- [Towards Accurate Binary Convolutional Neural Network](https://mlsanity.com/paper/1711.11294/) (2017)
- [Residual Feature Distillation Network for Lightweight Image Super-Resolution](https://mlsanity.com/paper/2009.11551/) (2020)
- [Fast and Accurate Single Image Super-Resolution via Information Distillation Network](https://mlsanity.com/paper/1803.09454/) (2018)
- [Deep Learning with Low Precision by Half-wave Gaussian Quantization](https://mlsanity.com/paper/1702.00953/) (2017)
- [Rethinking Architecture Selection in Differentiable NAS](https://mlsanity.com/paper/2108.04392/) (2021)
- [A Deep Journey into Super-resolution: A survey](https://mlsanity.com/paper/1904.07523/) (2019)
- [Rethinking Differentiable Search for Mixed-Precision Neural Networks](https://mlsanity.com/paper/2004.05795/) (2020)
- [AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks](https://mlsanity.com/paper/2006.08198/) (2020)
- [Differentiable Model Compression via Pseudo Quantization Noise](https://mlsanity.com/paper/2104.09987/) (2021)
- [A Survey on Model Compression and Acceleration for Pretrained Language Models](https://mlsanity.com/paper/2202.07105/) (2022)
- [AIM 2020 Challenge on Efficient Super-Resolution: Methods and Results](https://mlsanity.com/paper/2009.06943/) (2020)
- [ISTA-NAS: Efficient and Consistent Neural Architecture Search by Sparse Coding](https://mlsanity.com/paper/2010.06176/) (2020)
- [NIPQ: Noise proxy-based Integrated Pseudo-Quantization](https://mlsanity.com/paper/2206.00820/) (2022)
- [Real-Time Quantized Image Super-Resolution on Mobile NPUs, Mobile AI 2021 Challenge: Report](https://mlsanity.com/paper/2105.07825/) (2021)
- [RAISR: Rapid and Accurate Image Super Resolution](https://mlsanity.com/paper/1606.01299/) (2016)
- [BATS: Binary ArchitecTure Search](https://mlsanity.com/paper/2003.01711/) (2020)
- [Once Quantization-Aware Training: High Performance Extremely Low-bit Architecture Search](https://mlsanity.com/paper/2010.04354/) (2020)
- [Trilevel Neural Architecture Search for Efficient Single Image Super-Resolution](https://mlsanity.com/paper/2101.06658/) (2021)

## Built on by (most cited first, 4 of 4)

- [Investigating the Impact of Quantization Methods on the Safety and Reliability of Large Language Models](https://mlsanity.com/paper/2502.15799/) (2025)
- [SeqNAS: Neural Architecture Search for Event Sequence Classification](https://mlsanity.com/paper/2401.03246/) (2024)
- [GIFT-SW: Gaussian noise Injected Fine-Tuning of Salient Weights for LLMs](https://mlsanity.com/paper/2408.15300/) (2024)
- [Motivating Next-Gen Accelerators with Flexible (N:M) Activation Sparsity via Benchmarking Lightweight Post-Training Sparsification Approaches](https://mlsanity.com/paper/2509.22166/) (2025)

## Abstract (arXiv)

There is a constant need for high-performing and computationally efficient neural network models for image super-resolution: computationally efficient models can be used via low-capacity devices and reduce carbon footprints. One way to obtain such models is to compress models, e.g. quantization. Another way is a neural architecture search that automatically discovers new, more efficient solutions. We propose a novel quantization-aware procedure, the QuantNAS that combines pros of these two approaches. To make QuantNAS work, the procedure looks for quantization-friendly super-resolution models. The approach utilizes entropy regularization, quantization noise, and Adaptive Deviation for Quantization (ADQ) module to enhance the search procedure. The entropy regularization technique prioritizes a single operation within each block of the search space. Adding quantization noise to parameters and activations approximates model degradation after quantization, resulting in a more quantization-friendly architectures. ADQ helps to alleviate problems caused by Batch Norm blocks in super-resolution models. Our experimental results show that the proposed approximations are better for search procedure than direct model quantization. QuantNAS discovers architectures with better PSNR/BitOps trade-off than uniform or mixed precision quantization of fixed architectures. We showcase the effectiveness of our method through its application to two search spaces inspired by the state-of-the-art SR models and RFDN. Thus, anyone can design a proper search space based on an existing architecture and apply our method to obtain better quality and efficiency. The proposed procedure is 30% faster than direct weight quantization and is more stable.