# Real-Time Quantized Image Super-Resolution on Mobile NPUs, Mobile AI 2021 Challenge: Report

arXiv: https://arxiv.org/abs/2105.07825 · HTML: https://mlsanity.com/paper/2105.07825/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 60 references that are in the map.
- First posted on arXiv 17 May 2021.

## Builds on (most cited first, 30 of 31)

- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [RepVGG: Making VGG-style ConvNets Great Again](https://mlsanity.com/paper/2101.03697/) (2021)
- [FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.03443/) (2018)
- [ZeroQ: A Novel Zero Shot Quantization Framework](https://mlsanity.com/paper/2001.00281/) (2020)
- [Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithm](https://mlsanity.com/paper/1808.00278/) (2018)
- [MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning](https://mlsanity.com/paper/1903.10258/) (2019)
- [DSLR-Quality Photos on Mobile Devices with Deep Convolutional Networks](https://mlsanity.com/paper/1704.02470/) (2017)
- [Quantization Networks](https://mlsanity.com/paper/1911.09464/) (2019)
- [FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions](https://mlsanity.com/paper/2004.05565/) (2020)
- [Replacing Mobile Camera ISP with a Single Deep Learning Model](https://mlsanity.com/paper/2002.05509/) (2020)
- [Trained Quantization Thresholds for Accurate and Efficient Fixed-Point Inference of Deep Neural Networks](https://mlsanity.com/paper/1903.08066/) (2019)
- [NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Results](https://mlsanity.com/paper/2005.01996/) (2020)
- [AI Benchmark: All About Deep Learning on Smartphones in 2019](https://mlsanity.com/paper/1910.06663/) (2019)
- [AI Benchmark: Running Deep Neural Networks on Android Smartphones](https://mlsanity.com/paper/1810.01109/) (2018)
- [Learning Filter Basis for Convolutional Neural Network Compression](https://mlsanity.com/paper/1908.08932/) (2019)
- [Rendering Natural Camera Bokeh Effect with Deep Learning](https://mlsanity.com/paper/2006.05698/) (2020)
- [Seven ways to improve example-based single image super resolution](https://mlsanity.com/paper/1511.02228/) (2015)
- [NTIRE 2020 Challenge on Real Image Denoising: Dataset, Methods and Results](https://mlsanity.com/paper/2005.04117/) (2020)
- [AIM 2020 Challenge on Learned Image Signal Processing Pipeline](https://mlsanity.com/paper/2011.04994/) (2020)
- [PIRM Challenge on Perceptual Image Enhancement on Smartphones: Report](https://mlsanity.com/paper/1810.01641/) (2018)
- [WESPE: Weakly Supervised Photo Enhancer for Digital Cameras](https://mlsanity.com/paper/1709.01118/) (2017)
- [AIM 2020 Challenge on Rendering Realistic Bokeh](https://mlsanity.com/paper/2011.04988/) (2020)
- [T-Basis: a Compact Representation for Neural Networks](https://mlsanity.com/paper/2007.06631/) (2020)
- [NTIRE 2020 Challenge on Perceptual Extreme Super-Resolution: Methods and Results](https://mlsanity.com/paper/2005.01056/) (2020)
- [Deploying Image Deblurring across Mobile Devices: A Perspective of Quality and Latency](https://mlsanity.com/paper/2004.12599/) (2020)

## Built on by (most cited first, 30 of 34)

- [Residual Local Feature Network for Efficient Super-Resolution](https://mlsanity.com/paper/2205.07514/) (2022)
- [NILUT: Conditional Neural Implicit 3D Lookup Tables for Image Enhancement](https://mlsanity.com/paper/2306.11920/) (2023)
- [Extremely Lightweight Quantization Robust Real-Time Single-Image Super Resolution for Mobile Devices](https://mlsanity.com/paper/2105.10288/) (2021)
- [CADyQ: Content-Aware Dynamic Quantization for Image Super-Resolution](https://mlsanity.com/paper/2207.10345/) (2022)
- [FADE: Fusing the Assets of Decoder and Encoder for Task-Agnostic Upsampling](https://mlsanity.com/paper/2207.10392/) (2022)
- [Anchor-based Plain Net for Mobile Image Super-Resolution](https://mlsanity.com/paper/2105.09750/) (2021)
- [See More Details: Efficient Image Super-Resolution by Experts Mining](https://mlsanity.com/paper/2402.03412/) (2024)
- [Compiler-Aware Neural Architecture Search for On-Mobile Real-time Super-Resolution](https://mlsanity.com/paper/2207.12577/) (2022)
- [DIPNet: Efficiency Distillation and Iterative Pruning for Image Super-Resolution](https://mlsanity.com/paper/2304.07018/) (2023)
- [Low-field magnetic resonance image enhancement via stochastic image quality transfer](https://mlsanity.com/paper/2304.13385/) (2023)
- [Real-Time 4K Super-Resolution of Compressed AVIF Images. AIS 2024 Challenge Survey](https://mlsanity.com/paper/2404.16484/) (2024)
- [Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs, Mobile AI & AIM 2022 challenge: Report](https://mlsanity.com/paper/2211.05910/) (2022)
- [Learned Smartphone ISP on Mobile GPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report](https://mlsanity.com/paper/2211.03885/) (2022)
- [MicroISP: Processing 32MP Photos on Mobile Devices with Deep Learning](https://mlsanity.com/paper/2211.06770/) (2022)
- [edge-SR: Super-Resolution For The Masses](https://mlsanity.com/paper/2108.10335/) (2021)
- [CUF: Continuous Upsampling Filters](https://mlsanity.com/paper/2210.06965/) (2022)
- [AsConvSR: Fast and Lightweight Super-Resolution Network with Assembled Convolutions](https://mlsanity.com/paper/2305.03387/) (2023)
- [PyNet-V2 Mobile: Efficient On-Device Photo Processing With Neural Networks](https://mlsanity.com/paper/2211.06263/) (2022)
- [Fast Nearest Convolution for Real-Time Efficient Image Super-Resolution](https://mlsanity.com/paper/2208.11609/) (2022)
- [Bicubic++: Slim, Slimmer, Slimmest -- Designing an Industry-Grade Super-Resolution Network](https://mlsanity.com/paper/2305.02126/) (2023)
- [Overcoming Distribution Mismatch in Quantizing Image Super-Resolution Networks](https://mlsanity.com/paper/2307.13337/) (2023)
- [SYENet: A Simple Yet Effective Network for Multiple Low-Level Vision Tasks with Real-time Performance on Mobile Device](https://mlsanity.com/paper/2308.08137/) (2023)
- [QuantNAS for super resolution: searching for efficient quantization-friendly architectures against quantization noise](https://mlsanity.com/paper/2208.14839/) (2022)
- [Efficient Single-Image Depth Estimation on Mobile Devices, Mobile AI & AIM 2022 Challenge: Report](https://mlsanity.com/paper/2211.04470/) (2022)
- [Data Upcycling Knowledge Distillation for Image Super-Resolution](https://mlsanity.com/paper/2309.14162/) (2023)
- [Sliding Window Recurrent Network for Efficient Video Super-Resolution](https://mlsanity.com/paper/2208.11608/) (2022)
- [XCAT -- Lightweight Quantized Single Image Super-Resolution using Heterogeneous Group Convolutions and Cross Concatenation](https://mlsanity.com/paper/2208.14655/) (2022)
- [Power Efficient Video Super-Resolution on Mobile NPUs with Deep Learning, Mobile AI & AIM 2022 challenge: Report](https://mlsanity.com/paper/2211.05256/) (2022)
- [FADE: A Task-Agnostic Upsampling Operator for Encoder-Decoder Architectures](https://mlsanity.com/paper/2407.13500/) (2024)
- [Real-Time Super-Resolution for Real-World Images on Mobile Devices](https://mlsanity.com/paper/2206.01777/) (2022)

## Abstract (arXiv)

Image super-resolution is one of the most popular computer vision problems with many important applications to mobile devices. While many solutions have been proposed for this task, they are usually not optimized even for common smartphone AI hardware, not to mention more constrained smart TV platforms that are often supporting INT8 inference only. To address this problem, we introduce the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based image super-resolution solutions that can demonstrate a real-time performance on mobile or edge NPUs. For this, the participants were provided with the DIV2K dataset and trained quantized models to do an efficient 3X image upscaling. The runtime of all models was evaluated on the Synaptics VS680 Smart Home board with a dedicated NPU capable of accelerating quantized neural networks. The proposed solutions are fully compatible with all major mobile AI accelerators and are capable of reconstructing Full HD images under 40-60 ms while achieving high fidelity results. A detailed description of all models developed in the challenge is provided in this paper.