# Model-Based Image Signal Processors via Learnable Dictionaries

arXiv: https://arxiv.org/abs/2201.03210 · HTML: https://mlsanity.com/paper/2201.03210/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 45 references that are in the map.
- First posted on arXiv 10 Jan 2022.

## Builds on (most cited first, 20 of 20)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design](https://mlsanity.com/paper/1902.00275/) (2019)
- [Unprocessing Images for Learned Raw Denoising](https://mlsanity.com/paper/1811.11127/) (2018)
- [CycleISP: Real Image Restoration via Improved Data Synthesis](https://mlsanity.com/paper/2003.07761/) (2020)
- [Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration](https://mlsanity.com/paper/1508.02848/) (2015)
- [Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline](https://mlsanity.com/paper/2004.01179/) (2020)
- [Replacing Mobile Camera ISP with a Single Deep Learning Model](https://mlsanity.com/paper/2002.05509/) (2020)
- [Invertible Image Signal Processing](https://mlsanity.com/paper/2103.15061/) (2021)
- [DeepISP: Towards Learning an End-to-End Image Processing Pipeline](https://mlsanity.com/paper/1801.06724/) (2018)
- [Handheld Multi-Frame Super-Resolution](https://mlsanity.com/paper/1905.03277/) (2019)
- [Mobile Computational Photography: A Tour](https://mlsanity.com/paper/2102.09000/) (2021)
- [NTIRE 2020 Challenge on Real Image Denoising: Dataset, Methods and Results](https://mlsanity.com/paper/2005.04117/) (2020)
- [CameraNet: A Two-Stage Framework for Effective Camera ISP Learning](https://mlsanity.com/paper/1908.01481/) (2019)
- [Fast Fourier Color Constancy](https://mlsanity.com/paper/1611.07596/) (2016)
- [Reconfiguring the Imaging Pipeline for Computer Vision](https://mlsanity.com/paper/1705.04352/) (2017)
- [CIE XYZ Net: Unprocessing Images for Low-Level Computer Vision Tasks](https://mlsanity.com/paper/2006.12709/) (2020)
- [Fast and Accurate Single-Image Depth Estimation on Mobile Devices, Mobile AI 2021 Challenge: Report](https://mlsanity.com/paper/2105.08630/) (2021)
- [A Multi-Hypothesis Approach to Color Constancy](https://mlsanity.com/paper/2002.12896/) (2020)

## Built on by (most cited first, 30 of 43)

- [Swin2SR: SwinV2 Transformer for Compressed Image Super-Resolution and Restoration](https://mlsanity.com/paper/2209.11345/) (2022)
- [DarkIR: Robust Low-Light Image Restoration](https://mlsanity.com/paper/2412.13443/) (2024)
- [NILUT: Conditional Neural Implicit 3D Lookup Tables for Image Enhancement](https://mlsanity.com/paper/2306.11920/) (2023)
- [Unveiling Hidden Details: A RAW Data-Enhanced Paradigm for Real-World Super-Resolution](https://mlsanity.com/paper/2411.10798/) (2024)
- [DynamicISP: Dynamically Controlled Image Signal Processor for Image Recognition](https://mlsanity.com/paper/2211.01146/) (2022)
- [ParamISP: Learned Forward and Inverse ISPs using Camera Parameters](https://mlsanity.com/paper/2312.13313/) (2023)
- [RAW-Adapter: Adapting Pre-trained Visual Model to Camera RAW Images](https://mlsanity.com/paper/2408.14802/) (2024)
- [Perceptual Image Enhancement for Smartphone Real-Time Applications](https://mlsanity.com/paper/2210.13552/) (2022)
- [Efficient Visual Computing with Camera RAW Snapshots](https://mlsanity.com/paper/2212.07778/) (2022)
- [MIPI 2022 Challenge on Under-Display Camera Image Restoration: Methods and Results](https://mlsanity.com/paper/2209.07052/) (2022)
- [Learned Smartphone ISP on Mobile GPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report](https://mlsanity.com/paper/2211.03885/) (2022)
- [Reversed Image Signal Processing and RAW Reconstruction. AIM 2022 Challenge Report](https://mlsanity.com/paper/2210.11153/) (2022)
- [BSRAW: Improving Blind RAW Image Super-Resolution](https://mlsanity.com/paper/2312.15487/) (2023)
- [RAW-Diffusion: RGB-Guided Diffusion Models for High-Fidelity RAW Image Generation](https://mlsanity.com/paper/2411.13150/) (2024)
- [Toward Efficient Deep Blind RAW Image Restoration](https://mlsanity.com/paper/2409.18204/) (2024)
- [Deep RAW Image Super-Resolution. A NTIRE 2024 Challenge Survey](https://mlsanity.com/paper/2404.16223/) (2024)
- [Learning Task-Oriented Flows to Mutually Guide Feature Alignment in Synthesized and Real Video Denoising](https://mlsanity.com/paper/2208.11803/) (2022)
- [Real-Time Under-Display Cameras Image Restoration and HDR on Mobile Devices](https://mlsanity.com/paper/2211.14040/) (2022)
- [Self-Supervised Reversed Image Signal Processing via Reference-Guided Dynamic Parameter Selection](https://mlsanity.com/paper/2303.13916/) (2023)
- [Unsupervised Low-light Image Enhancement with Lookup Tables and Diffusion Priors](https://mlsanity.com/paper/2409.18899/) (2024)
- [Mixture-of-Shape-Experts (MoSE): End-to-End Shape Dictionary Framework to Prompt SAM for Generalizable Medical Segmentation](https://mlsanity.com/paper/2504.09601/) (2025)
- [RAW Image Reconstruction from RGB on Smartphones. NTIRE 2025 Challenge Report](https://mlsanity.com/paper/2506.01947/) (2025)
- [RAW-Flow: Advancing RGB-to-RAW Image Reconstruction with Deterministic Latent Flow Matching](https://mlsanity.com/paper/2601.20364/) (2026)
- [Reversing Image Signal Processors by Reverse Style Transferring](https://mlsanity.com/paper/2210.09074/) (2022)
- [CSwin2SR: Circular Swin2SR for Compressed Image Super-Resolution](https://mlsanity.com/paper/2301.08749/) (2023)
- [A Learnable Color Correction Matrix for RAW Reconstruction](https://mlsanity.com/paper/2409.02497/) (2024)
- [RAWMamba: Unified sRGB-to-RAW De-rendering With State Space Model](https://mlsanity.com/paper/2411.11717/) (2024)
- [PQDynamicISP: Dynamically Controlled Image Signal Processor for Any Image Sensors Pursuing Perceptual Quality](https://mlsanity.com/paper/2403.10091/) (2024)
- [Simple Image Signal Processing using Global Context Guidance](https://mlsanity.com/paper/2404.11569/) (2024)
- [RAW-Adapter: Adapting Pre-trained Visual Model to Camera RAW Images and A Benchmark](https://mlsanity.com/paper/2503.17027/) (2025)

## Abstract (arXiv)

Digital cameras transform sensor RAW readings into RGB images by means of their Image Signal Processor (ISP). Computational photography tasks such as image denoising and colour constancy are commonly performed in the RAW domain, in part due to the inherent hardware design, but also due to the appealing simplicity of noise statistics that result from the direct sensor readings. Despite this, the availability of RAW images is limited in comparison with the abundance and diversity of available RGB data. Recent approaches have attempted to bridge this gap by estimating the RGB to RAW mapping: handcrafted model-based methods that are interpretable and controllable usually require manual parameter fine-tuning, while end-to-end learnable neural networks require large amounts of training data, at times with complex training procedures, and generally lack interpretability and parametric control. Towards addressing these existing limitations, we present a novel hybrid model-based and data-driven ISP that builds on canonical ISP operations and is both learnable and interpretable. Our proposed invertible model, capable of bidirectional mapping between RAW and RGB domains, employs end-to-end learning of rich parameter representations, i.e. dictionaries, that are free from direct parametric supervision and additionally enable simple and plausible data augmentation. We evidence the value of our data generation process by extensive experiments under both RAW image reconstruction and RAW image denoising tasks, obtaining state-of-the-art performance in both. Additionally, we show that our ISP can learn meaningful mappings from few data samples, and that denoising models trained with our dictionary-based data augmentation are competitive despite having only few or zero ground-truth labels.