# Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Report

arXiv: https://arxiv.org/abs/2105.07809 · HTML: https://mlsanity.com/paper/2105.07809/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 56 references that are in the map.
- First posted on arXiv 17 May 2021.

## Builds on (most cited first, 30 of 33)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [Once-for-All: Train One Network and Specialize it for Efficient Deployment](https://mlsanity.com/paper/1908.09791/) (2019)
- [FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.03443/) (2018)
- [Accelerating the Super-Resolution Convolutional Neural Network](https://mlsanity.com/paper/1608.00367/) (2016)
- [ZeroQ: A Novel Zero Shot Quantization Framework](https://mlsanity.com/paper/2001.00281/) (2020)
- [Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithm](https://mlsanity.com/paper/1808.00278/) (2018)
- [MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning](https://mlsanity.com/paper/1903.10258/) (2019)
- [DSLR-Quality Photos on Mobile Devices with Deep Convolutional Networks](https://mlsanity.com/paper/1704.02470/) (2017)
- [Quantization Networks](https://mlsanity.com/paper/1911.09464/) (2019)
- [FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions](https://mlsanity.com/paper/2004.05565/) (2020)
- [Learning Accurate Dense Correspondences and When to Trust Them](https://mlsanity.com/paper/2101.01710/) (2021)
- [Replacing Mobile Camera ISP with a Single Deep Learning Model](https://mlsanity.com/paper/2002.05509/) (2020)
- [Trained Quantization Thresholds for Accurate and Efficient Fixed-Point Inference of Deep Neural Networks](https://mlsanity.com/paper/1903.08066/) (2019)
- [NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Results](https://mlsanity.com/paper/2005.01996/) (2020)
- [AI Benchmark: All About Deep Learning on Smartphones in 2019](https://mlsanity.com/paper/1910.06663/) (2019)
- [AI Benchmark: Running Deep Neural Networks on Android Smartphones](https://mlsanity.com/paper/1810.01109/) (2018)
- [Unsupervised Learning for Real-World Super-Resolution](https://mlsanity.com/paper/1909.09629/) (2019)
- [Learning Filter Basis for Convolutional Neural Network Compression](https://mlsanity.com/paper/1908.08932/) (2019)
- [Rendering Natural Camera Bokeh Effect with Deep Learning](https://mlsanity.com/paper/2006.05698/) (2020)
- [AIM 2020 Challenge on Learned Image Signal Processing Pipeline](https://mlsanity.com/paper/2011.04994/) (2020)
- [PIRM Challenge on Perceptual Image Enhancement on Smartphones: Report](https://mlsanity.com/paper/1810.01641/) (2018)
- [AWNet: Attentive Wavelet Network for Image ISP](https://mlsanity.com/paper/2008.09228/) (2020)
- [WESPE: Weakly Supervised Photo Enhancer for Digital Cameras](https://mlsanity.com/paper/1709.01118/) (2017)
- [AIM 2020 Challenge on Rendering Realistic Bokeh](https://mlsanity.com/paper/2011.04988/) (2020)
- [T-Basis: a Compact Representation for Neural Networks](https://mlsanity.com/paper/2007.06631/) (2020)
- [Deploying Image Deblurring across Mobile Devices: A Perspective of Quality and Latency](https://mlsanity.com/paper/2004.12599/) (2020)

## Built on by (most cited first, 30 of 34)

- [NILUT: Conditional Neural Implicit 3D Lookup Tables for Image Enhancement](https://mlsanity.com/paper/2306.11920/) (2023)
- [Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs, Mobile AI & AIM 2022 challenge: Report](https://mlsanity.com/paper/2211.05910/) (2022)
- [Enhancing RAW-to-sRGB with Decoupled Style Structure in Fourier Domain](https://mlsanity.com/paper/2401.02161/) (2024)
- [Learned Smartphone ISP on Mobile GPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report](https://mlsanity.com/paper/2211.03885/) (2022)
- [MicroISP: Processing 32MP Photos on Mobile Devices with Deep Learning](https://mlsanity.com/paper/2211.06770/) (2022)
- [MobileIE: An Extremely Lightweight and Effective ConvNet for Real-Time Image Enhancement on Mobile Devices](https://mlsanity.com/paper/2507.01838/) (2025)
- [BSRAW: Improving Blind RAW Image Super-Resolution](https://mlsanity.com/paper/2312.15487/) (2023)
- [Rawformer: Unpaired Raw-to-Raw Translation for Learnable Camera ISPs](https://mlsanity.com/paper/2404.10700/) (2024)
- [NTIRE 2025 Challenge on RAW Image Restoration and Super-Resolution](https://mlsanity.com/paper/2506.02197/) (2025)
- [PyNet-V2 Mobile: Efficient On-Device Photo Processing With Neural Networks](https://mlsanity.com/paper/2211.06263/) (2022)
- [ISPDiffuser: Learning RAW-to-sRGB Mappings with Texture-Aware Diffusion Models and Histogram-Guided Color Consistency](https://mlsanity.com/paper/2503.19283/) (2025)
- [Deep RAW Image Super-Resolution. A NTIRE 2024 Challenge Survey](https://mlsanity.com/paper/2404.16223/) (2024)
- [SYENet: A Simple Yet Effective Network for Multiple Low-Level Vision Tasks with Real-time Performance on Mobile Device](https://mlsanity.com/paper/2308.08137/) (2023)
- [MetaISP -- Exploiting Global Scene Structure for Accurate Multi-Device Color Rendition](https://mlsanity.com/paper/2401.03220/) (2024)
- [Efficient Single-Image Depth Estimation on Mobile Devices, Mobile AI & AIM 2022 Challenge: Report](https://mlsanity.com/paper/2211.04470/) (2022)
- [ISP-Agnostic Image Reconstruction for Under-Display Cameras](https://mlsanity.com/paper/2111.01511/) (2021)
- [Reversing Image Signal Processors by Reverse Style Transferring](https://mlsanity.com/paper/2210.09074/) (2022)
- [Power Efficient Video Super-Resolution on Mobile NPUs with Deep Learning, Mobile AI & AIM 2022 challenge: Report](https://mlsanity.com/paper/2211.05256/) (2022)
- [Learned Lightweight Smartphone ISP with Unpaired Data](https://mlsanity.com/paper/2505.10420/) (2025)
- [Realistic Bokeh Effect Rendering on Mobile GPUs, Mobile AI & AIM 2022 challenge: Report](https://mlsanity.com/paper/2211.06769/) (2022)
- [PQDynamicISP: Dynamically Controlled Image Signal Processor for Any Image Sensors Pursuing Perceptual Quality](https://mlsanity.com/paper/2403.10091/) (2024)
- [Simple Image Signal Processing using Global Context Guidance](https://mlsanity.com/paper/2404.11569/) (2024)
- [CRISPnet: Color Rendition ISP Net](https://mlsanity.com/paper/2203.10562/) (2022)
- [LW-ISP: A Lightweight Model with ISP and Deep Learning](https://mlsanity.com/paper/2210.03904/) (2022)
- [Overexposure Mask Fusion: Generalizable Reverse ISP Multi-Step Refinement](https://mlsanity.com/paper/2210.11511/) (2022)
- [ISP meets Deep Learning: A Survey on Deep Learning Methods for Image Signal Processing](https://mlsanity.com/paper/2305.11994/) (2023)
- [Enhanced Super-Resolution Training via Mimicked Alignment for Real-World Scenes](https://mlsanity.com/paper/2410.05410/) (2024)
- [Experts-Guided Unbalanced Optimal Transport for ISP Learning from Unpaired and/or Paired Data](https://mlsanity.com/paper/2512.05635/) (2025)
- [Addressing Image Authenticity When Cameras Use Generative AI](https://mlsanity.com/paper/2604.21879/) (2026)
- [Fully Quantized Always-on Face Detector Considering Mobile Image Sensors](https://mlsanity.com/paper/2311.01001/) (2023)

## Abstract (arXiv)

As the quality of mobile cameras starts to play a crucial role in modern smartphones, more and more attention is now being paid to ISP algorithms used to improve various perceptual aspects of mobile photos. In this Mobile AI challenge, the target was to develop an end-to-end deep learning-based image signal processing (ISP) pipeline that can replace classical hand-crafted ISPs and achieve nearly real-time performance on smartphone NPUs. For this, the participants were provided with a novel learned ISP dataset consisting of RAW-RGB image pairs captured with the Sony IMX586 Quad Bayer mobile sensor and a professional 102-megapixel medium format camera. The runtime of all models was evaluated on the MediaTek Dimensity 1000+ platform with a dedicated AI processing unit capable of accelerating both floating-point and quantized neural networks. The proposed solutions are fully compatible with the above NPU and are capable of processing Full HD photos under 60-100 milliseconds while achieving high fidelity results. A detailed description of all models developed in this challenge is provided in this paper.