mlsanityData to 7 Oct 2026

Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Report

arXiv 2105.07809 ↗First posted 17 May 2021eess.IV cs.CV cs.LGcited by 34

Not on the map. Why?

Abstract (arXiv)

As the quality of mobile cameras starts to play a crucial role in modern smartphones, more and more attention is now being paid to ISP algorithms used to improve various perceptual aspects of mobile photos. In this Mobile AI challenge, the target was to develop an end-to-end deep learning-based image signal processing (ISP) pipeline that can replace classical hand-crafted ISPs and achieve nearly real-time performance on smartphone NPUs. For this, the participants were provided with a novel learned ISP dataset consisting of RAW-RGB image pairs captured with the Sony IMX586 Quad Bayer mobile sensor and a professional 102-megapixel medium format camera. The runtime of all models was evaluated on the MediaTek Dimensity 1000+ platform with a dedicated AI processing unit capable of accelerating both floating-point and quantized neural networks. The proposed solutions are fully compatible with the above NPU and are capable of processing Full HD photos under 60-100 milliseconds while achieving high fidelity results. A detailed description of all models developed in this challenge is provided in this paper.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 33 of its 56 references that are in the map. First posted on arXiv 17 May 2021.

Builds on

33 of 56 references.

References · 33

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2019Searching for MobileNetV3cited by 1,984
  3. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  4. 2018ESRGAN: Enhanced Super-Resolution Generative Adversarial Networkscited by 1,250
  5. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  6. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 778
  7. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 480
  8. 2016Accelerating the Super-Resolution Convolutional Neural Networkcited by 393
  9. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 265
  10. 2018Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithmcited by 233
  11. 2019MetaPruning: Meta Learning for Automatic Neural Network Channel Pruningcited by 226
  12. 2017DSLR-Quality Photos on Mobile Devices with Deep Convolutional Networkscited by 156
21 more
  1. 2020FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensionscited by 152
  2. 2019Quantization Networkscited by 151
  3. 2021Learning Accurate Dense Correspondences and When to Trust Themcited by 113
  4. 2020Replacing Mobile Camera ISP with a Single Deep Learning Modelcited by 108
  5. 2019Trained Quantization Thresholds for Accurate and Efficient Fixed-Point Inference of Deep Neural Networkscited by 87
  6. 2020NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Resultscited by 83
  7. 2019AI Benchmark: All About Deep Learning on Smartphones in 2019cited by 76
  8. 2018AI Benchmark: Running Deep Neural Networks on Android Smartphonescited by 75
  9. 2019Unsupervised Learning for Real-World Super-Resolutioncited by 64
  10. 2019Learning Filter Basis for Convolutional Neural Network Compressioncited by 57
  11. 2020Rendering Natural Camera Bokeh Effect with Deep Learningcited by 57
  12. 2020AIM 2020 Challenge on Learned Image Signal Processing Pipelinecited by 44
  13. 2018PIRM Challenge on Perceptual Image Enhancement on Smartphones: Reportcited by 38
  14. 2020AWNet: Attentive Wavelet Network for Image ISPcited by 38
  15. 2017WESPE: Weakly Supervised Photo Enhancer for Digital Camerascited by 32
  16. 2020AIM 2020 Challenge on Rendering Realistic Bokehcited by 26
  17. 2020T-Basis: a Compact Representation for Neural Networkscited by 21
  18. 2020Deploying Image Deblurring across Mobile Devices: A Perspective of Quality and Latencycited by 16
  19. 2020Controlling Information Capacity of Binary Neural Networkcited by 15
  20. 2021PyNET-CA: Enhanced PyNET with Channel Attention for End-to-End Mobile Image Signal Processingcited by 13
  21. 2018Fast Perceptual Image Enhancementcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 11; 2023: 5; 2024: 7; 2025: 6; 2026: 4.

34 papers.

Sort:

By area:

  1. 2023NILUT: Conditional Neural Implicit 3D Lookup Tables for Image Enhancementcited by 31
  2. 2022Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs, Mobile AI & AIM 2022 challenge: Reportcited by 16
  3. 2024Enhancing RAW-to-sRGB with Decoupled Style Structure in Fourier Domaincited by 14
  4. 2022Learned Smartphone ISP on Mobile GPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Reportcited by 13
  5. 2022MicroISP: Processing 32MP Photos on Mobile Devices with Deep Learningcited by 13
  6. 2025MobileIE: An Extremely Lightweight and Effective ConvNet for Real-Time Image Enhancement on Mobile Devicescited by 12
  7. 2023BSRAW: Improving Blind RAW Image Super-Resolutioncited by 11
  8. 2024Rawformer: Unpaired Raw-to-Raw Translation for Learnable Camera ISPscited by 11
  9. 2025NTIRE 2025 Challenge on RAW Image Restoration and Super-Resolutioncited by 9
  10. 2022PyNet-V2 Mobile: Efficient On-Device Photo Processing With Neural Networkscited by 7
  11. 2025ISPDiffuser: Learning RAW-to-sRGB Mappings with Texture-Aware Diffusion Models and Histogram-Guided Color Consistencycited by 7
  12. 2024Deep RAW Image Super-Resolution. A NTIRE 2024 Challenge Surveycited by 6
22 more
  1. 2023SYENet: A Simple Yet Effective Network for Multiple Low-Level Vision Tasks with Real-time Performance on Mobile Devicecited by 5
  2. 2024MetaISP -- Exploiting Global Scene Structure for Accurate Multi-Device Color Renditioncited by 5
  3. 2022Efficient Single-Image Depth Estimation on Mobile Devices, Mobile AI & AIM 2022 Challenge: Reportcited by 4
  4. 2021ISP-Agnostic Image Reconstruction for Under-Display Camerascited by 3
  5. 2022Reversing Image Signal Processors by Reverse Style Transferringcited by 3
  6. 2022Power Efficient Video Super-Resolution on Mobile NPUs with Deep Learning, Mobile AI & AIM 2022 challenge: Reportcited by 3
  7. 2025Learned Lightweight Smartphone ISP with Unpaired Datacited by 3
  8. 2022Realistic Bokeh Effect Rendering on Mobile GPUs, Mobile AI & AIM 2022 challenge: Reportcited by 2
  9. 2024PQDynamicISP: Dynamically Controlled Image Signal Processor for Any Image Sensors Pursuing Perceptual Qualitycited by 2
  10. 2024Simple Image Signal Processing using Global Context Guidancecited by 2
  11. 2022CRISPnet: Color Rendition ISP Netcited by 1
  12. 2022LW-ISP: A Lightweight Model with ISP and Deep Learningcited by 1
  13. 2022Overexposure Mask Fusion: Generalizable Reverse ISP Multi-Step Refinementcited by 1
  14. 2023ISP meets Deep Learning: A Survey on Deep Learning Methods for Image Signal Processingcited by 1
  15. 2024Enhanced Super-Resolution Training via Mimicked Alignment for Real-World Scenescited by 1
  16. 2025Experts-Guided Unbalanced Optimal Transport for ISP Learning from Unpaired and/or Paired Datacited by 1
  17. 2026Addressing Image Authenticity When Cameras Use Generative AIcited by 1
  18. 2023Fully Quantized Always-on Face Detector Considering Mobile Image Sensorscited by 0
  19. 2025Spatial-Frequency Aware for Object Detection in RAW Imagecited by 0
  20. 2026UHD Low-Light Image Enhancement via Real-Time Enhancement Methods with Clifford Information Fusioncited by 0
  21. 2026RPBA-Net: An Interpretable Residual Pyramid Bilateral Affine Network for RAW-Domain ISP Enhancementcited by 0
  22. 2026Visual Autoregressive Priors for RAW-to-sRGB Image Signal Processingcited by 0