Power Efficient Video Super-Resolution on Mobile NPUs with Deep Learning, Mobile AI & AIM 2022 challenge: Report
Not on the map. Why?
Abstract (arXiv)
Video super-resolution is one of the most popular tasks on mobile devices, being widely used for an automatic improvement of low-bitrate and low-resolution video streams. While numerous solutions have been proposed for this problem, they are usually quite computationally demanding, demonstrating low FPS rates and power efficiency on mobile devices. In this Mobile AI challenge, we address this problem and propose the participants to design an end-to-end real-time video super-resolution solution for mobile NPUs optimized for low energy consumption. The participants were provided with the REDS training dataset containing video sequences for a 4X video upscaling task. The runtime and power efficiency of all models was evaluated on the powerful MediaTek Dimensity 9000 platform with a dedicated AI processing unit capable of accelerating floating-point and quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 500 FPS rate and 0.2 [Watt / 30 FPS] power consumption. A detailed description of all models developed in the challenge is provided in this paper.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 51 of its 79 references that are in the map. First posted on arXiv 7 Nov 2022.
Builds on
51 of 79 references.
References · 51
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2019Searching for MobileNetV3cited by 1,984
- 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
- 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,199
- 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
- 2015Accurate Image Super-Resolution Using Very Deep Convolutional Networkscited by 902
- 2019EDVR: Video Restoration with Enhanced Deformable Convolutional Networkscited by 492
- 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 480
- 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 265
- 2018Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithmcited by 233
- 2019Lightweight Image Super-Resolution with Information Multi-distillation Networkcited by 227
38 more
- 2019MetaPruning: Meta Learning for Automatic Neural Network Channel Pruningcited by 226
- 2022VRT: A Video Restoration Transformercited by 219
- 2018Frame-Recurrent Video Super-Resolutioncited by 160
- 2017DSLR-Quality Photos on Mobile Devices with Deep Convolutional Networkscited by 156
- 2020FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensionscited by 152
- 2019Quantization Networkscited by 151
- 2020Residual Feature Distillation Network for Lightweight Image Super-Resolutioncited by 122
- 2020Replacing Mobile Camera ISP with a Single Deep Learning Modelcited by 108
- 2019Trained Quantization Thresholds for Accurate and Efficient Fixed-Point Inference of Deep Neural Networkscited by 87
- 2020NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Resultscited by 83
- 2019AI Benchmark: All About Deep Learning on Smartphones in 2019cited by 76
- 2018AI Benchmark: Running Deep Neural Networks on Android Smartphonescited by 75
- 2019Efficient Video Super-Resolution through Recurrent Latent Space Propagationcited by 69
- 2019Learning Filter Basis for Convolutional Neural Network Compressioncited by 57
- 2020Rendering Natural Camera Bokeh Effect with Deep Learningcited by 57
- 2020Video Super Resolution Based on Deep Learning: A Comprehensive Surveycited by 54
- 2020NTIRE 2020 Challenge on Real Image Denoising: Dataset, Methods and Resultscited by 51
- 2020Revisiting Temporal Modeling for Video Super-resolutioncited by 46
- 2020AIM 2020 Challenge on Learned Image Signal Processing Pipelinecited by 44
- 2022NTIRE 2022 Challenge on Efficient Super-Resolution: Methods and Resultscited by 40
- 2018PIRM Challenge on Perceptual Image Enhancement on Smartphones: Reportcited by 38
- 2021Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Reportcited by 34
- 2021Real-Time Quantized Image Super-Resolution on Mobile NPUs, Mobile AI 2021 Challenge: Reportcited by 34
- 2017WESPE: Weakly Supervised Photo Enhancer for Digital Camerascited by 32
- 2022BSRT: Improving Burst Super-Resolution with Swin Transformer and Flow-Guided Deformable Alignmentcited by 32
- 2020AIM 2020 Challenge on Rendering Realistic Bokehcited by 26
- 2021Anchor-based Plain Net for Mobile Image Super-Resolutioncited by 24
- 2020T-Basis: a Compact Representation for Neural Networkscited by 21
- 2022AIM 2022 Challenge on Super-Resolution of Compressed Image and Video: Dataset, Methods and Resultscited by 20
- 2021Fast and Accurate Single-Image Depth Estimation on Mobile Devices, Mobile AI 2021 Challenge: Reportcited by 19
- 2021Real-Time Video Super-Resolution on Smartphones with Deep Learning, Mobile AI 2021 Challenge: Reportcited by 19
- 2020Deploying Image Deblurring across Mobile Devices: A Perspective of Quality and Latencycited by 16
- 2020Controlling Information Capacity of Binary Neural Networkcited by 15
- 2021Fast Camera Image Denoising on Mobile GPUs with Deep Learning, Mobile AI 2021 Challenge: Reportcited by 15
- 2020NTIRE 2020 Challenge on Image and Video Deblurringcited by 11
- 2022Reversed Image Signal Processing and RAW Reconstruction. AIM 2022 Challenge Reportcited by 11
- 2021Fast and Accurate Quantized Camera Scene Detection on Smartphones, Mobile AI 2021 Challenge: Reportcited by 10
- 2022AIM 2022 Challenge on Instagram Filter Removal: Methods and Resultscited by 6
50 most cited of 51 All 51 (JSON)
Built on by
Citing papers per year: 2024: 2; 2025: 1.
3 papers.