# MetaISP -- Exploiting Global Scene Structure for Accurate Multi-Device Color Rendition

arXiv: https://arxiv.org/abs/2401.03220 · HTML: https://mlsanity.com/paper/2401.03220/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 45 references that are in the map.
- First posted on arXiv 6 Jan 2024.

## Builds on (most cited first, 30 of 32)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [Going deeper with Image Transformers](https://mlsanity.com/paper/2103.17239/) (2021)
- [Learning to See in the Dark](https://mlsanity.com/paper/1805.01934/) (2018)
- [XCiT: Cross-Covariance Image Transformers](https://mlsanity.com/paper/2106.09681/) (2021)
- [Toward Multimodal Image-to-Image Translation](https://mlsanity.com/paper/1711.11586/) (2017)
- [Learning Representations for Automatic Colorization](https://mlsanity.com/paper/1603.06668/) (2016)
- [Unprocessing Images for Learned Raw Denoising](https://mlsanity.com/paper/1811.11127/) (2018)
- [TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions](https://mlsanity.com/paper/2111.14813/) (2021)
- [CycleISP: Real Image Restoration via Improved Data Synthesis](https://mlsanity.com/paper/2003.07761/) (2020)
- [UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss](https://mlsanity.com/paper/1711.07837/) (2017)
- [Arbitrary Style Transfer with Style-Attentional Networks](https://mlsanity.com/paper/1812.02342/) (2018)
- [AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer](https://mlsanity.com/paper/2108.03647/) (2021)
- [Replacing Mobile Camera ISP with a Single Deep Learning Model](https://mlsanity.com/paper/2002.05509/) (2020)
- [Invertible Image Signal Processing](https://mlsanity.com/paper/2103.15061/) (2021)
- [You Only Need 90K Parameters to Adapt Light: A Light Weight Transformer for Image Enhancement and Exposure Correction](https://mlsanity.com/paper/2205.14871/) (2022)
- [Polarized Reflection Removal with Perfect Alignment in the Wild](https://mlsanity.com/paper/2003.12789/) (2020)
- [AIM 2020 Challenge on Learned Image Signal Processing Pipeline](https://mlsanity.com/paper/2011.04994/) (2020)
- [Learning RAW-to-sRGB Mappings with Inaccurately Aligned Supervision](https://mlsanity.com/paper/2108.08119/) (2021)

## Built on by (most cited first, 5 of 5)

- [RAW-Adapter: Adapting Pre-trained Visual Model to Camera RAW Images](https://mlsanity.com/paper/2408.14802/) (2024)
- [Retinex-RAWMamba: Bridging Demosaicing and Denoising for Low-Light RAW Image Enhancement](https://mlsanity.com/paper/2409.07040/) (2024)
- [RAW-Adapter: Adapting Pre-trained Visual Model to Camera RAW Images and A Benchmark](https://mlsanity.com/paper/2503.17027/) (2025)
- [Uni-ISP: Toward Unifying the Learning of ISPs from Multiple Mobile Cameras](https://mlsanity.com/paper/2406.01003/) (2024)
- [Lightweight Unpaired Smartphone ISP Transfer with Semantic Pseudo-Pairing](https://mlsanity.com/paper/2605.07495/) (2026)

## Abstract (arXiv)

Image signal processors (ISPs) are historically grown legacy software systems for reconstructing color images from noisy raw sensor measurements. Each smartphone manufacturer has developed its ISPs with its own characteristic heuristics for improving the color rendition, for example, skin tones and other visually essential colors. The recent interest in replacing the historically grown ISP systems with deep-learned pipelines to match DSLR's image quality improves structural features in the image. However, these works ignore the superior color processing based on semantic scene analysis that distinguishes mobile phone ISPs from DSLRs. Here, we present MetaISP, a single model designed to learn how to translate between the color and local contrast characteristics of different devices. MetaISP takes the RAW image from device A as input and translates it to RGB images that inherit the appearance characteristics of devices A, B, and C. We achieve this result by employing a lightweight deep learning technique that conditions its output appearance based on the device of interest. In this approach, we leverage novel attention mechanisms inspired by cross-covariance to learn global scene semantics. Additionally, we use the metadata that typically accompanies RAW images and estimate scene illuminants when they are unavailable.