# ISP Distillation

arXiv: https://arxiv.org/abs/2101.10203 · HTML: https://mlsanity.com/paper/2101.10203/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 38 references that are in the map.
- First posted on arXiv 25 Jan 2021.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Learning to See in the Dark](https://mlsanity.com/paper/1805.01934/) (2018)
- [Improving the Robustness of Deep Neural Networks via Stability Training](https://mlsanity.com/paper/1604.04326/) (2016)
- [DeepISP: Towards Learning an End-to-End Image Processing Pipeline](https://mlsanity.com/paper/1801.06724/) (2018)
- [Label Refinery: Improving ImageNet Classification through Label Progression](https://mlsanity.com/paper/1805.02641/) (2018)
- [Dirty Pixels: Towards End-to-End Image Processing and Perception](https://mlsanity.com/paper/1701.06487/) (2017)
- [Reconfiguring the Imaging Pipeline for Computer Vision](https://mlsanity.com/paper/1705.04352/) (2017)
- [VisionISP: Repurposing the Image Signal Processor for Computer Vision Applications](https://mlsanity.com/paper/1911.05931/) (2019)
- [Dynamic Low-light Imaging with Quanta Image Sensors](https://mlsanity.com/paper/2007.08614/) (2020)
- [DeepCorrect: Correcting DNN models against Image Distortions](https://mlsanity.com/paper/1705.02406/) (2017)
- [Classification Driven Dynamic Image Enhancement](https://mlsanity.com/paper/1710.07558/) (2017)
- [Image Classification in the Dark using Quanta Image Sensors](https://mlsanity.com/paper/2006.02026/) (2020)
- [ISP4ML: Understanding the Role of Image Signal Processing in Efficient Deep Learning Vision Systems](https://mlsanity.com/paper/1911.07954/) (2019)
- [Generative Sensing: Transforming Unreliable Sensor Data for Reliable Recognition](https://mlsanity.com/paper/1801.02684/) (2018)
- [Focus-and-Expand: Training Guidance Through Gradual Manipulation of Input Features](https://mlsanity.com/paper/2007.07723/) (2020)

## Built on by (most cited first, 7 of 7)

- [Learning RAW-to-sRGB Mappings with Inaccurately Aligned Supervision](https://mlsanity.com/paper/2108.08119/) (2021)
- [DynamicISP: Dynamically Controlled Image Signal Processor for Image Recognition](https://mlsanity.com/paper/2211.01146/) (2022)
- [Rawgment: Noise-Accounted RAW Augmentation Enables Recognition in a Wide Variety of Environments](https://mlsanity.com/paper/2210.16046/) (2022)
- [Beyond RGB: Adaptive Parallel Processing for RAW Object Detection](https://mlsanity.com/paper/2503.13163/) (2025)
- [Student-Teacher Learning from Clean Inputs to Noisy Inputs](https://mlsanity.com/paper/2103.07600/) (2021)
- [Enabling ISP-less Low-Power Computer Vision](https://mlsanity.com/paper/2210.05451/) (2022)
- [R3: Reconstruction, Raw, and Rain: Deraining Directly in the Bayer Domain](https://mlsanity.com/paper/2509.24022/) (2025)

## Abstract (arXiv)

Nowadays, many of the images captured are `observed' by machines only and not by humans, e.g., in autonomous systems. High-level machine vision models, such as object recognition or semantic segmentation, assume images are transformed into some canonical image space by the camera ansImage Signal Processor (ISP). However, the camera ISP is optimized for producing visually pleasing images for human observers and not for machines. Therefore, one may spare the ISP compute time and apply vision models directly to RAW images. Yet, it has been shown that training such models directly on RAW images results in a performance drop. To mitigate this drop, we use a RAW and RGB image pairs dataset, which can be easily acquired with no human labeling. We then train a model that is applied directly to the RAW data by using knowledge distillation such that the model predictions for RAW images will be aligned with the predictions of an off-the-shelf pre-trained model for processed RGB images. Our experiments show that our performance on RAW images for object classification and semantic segmentation is significantly better than models trained on labeled RAW images. It also reasonably matches the predictions of a pre-trained model on processed RGB images, while saving the ISP compute overhead.