# AdaptiveISP: Learning an Adaptive Image Signal Processor for Object Detection

arXiv: https://arxiv.org/abs/2410.22939 · HTML: https://mlsanity.com/paper/2410.22939/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 41 references that are in the map.
- First posted on arXiv 30 Oct 2024.

## Builds on (most cited first, 24 of 24)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [Unprocessing Images for Learned Raw Denoising](https://mlsanity.com/paper/1811.11127/) (2018)
- [CycleISP: Real Image Restoration via Improved Data Synthesis](https://mlsanity.com/paper/2003.07761/) (2020)
- [Replacing Mobile Camera ISP with a Single Deep Learning Model](https://mlsanity.com/paper/2002.05509/) (2020)
- [Image-Adaptive YOLO for Object Detection in Adverse Weather Conditions](https://mlsanity.com/paper/2112.08088/) (2021)
- [Dense Distinct Query for End-to-End Object Detection](https://mlsanity.com/paper/2303.12776/) (2023)
- [Distort-and-Recover: Color Enhancement using Deep Reinforcement Learning](https://mlsanity.com/paper/1804.04450/) (2018)
- [Unpaired Image Enhancement Featuring Reinforcement-Learning-Controlled Image Editing Software](https://mlsanity.com/paper/1912.07833/) (2019)
- [ReconfigISP: Reconfigurable Camera Image Processing Pipeline](https://mlsanity.com/paper/2109.04760/) (2021)
- [DynamicISP: Dynamically Controlled Image Signal Processor for Image Recognition](https://mlsanity.com/paper/2211.01146/) (2022)
- [ParamISP: Learned Forward and Inverse ISPs using Camera Parameters](https://mlsanity.com/paper/2312.13313/) (2023)
- [Automatic ISP image quality tuning using non-linear optimization](https://mlsanity.com/paper/1902.09023/) (2019)
- [VisionISP: Repurposing the Image Signal Processor for Computer Vision Applications](https://mlsanity.com/paper/1911.05931/) (2019)
- [Raw or Cooked? Object Detection on RAW Images](https://mlsanity.com/paper/2301.08965/) (2023)
- [PixelRL: Fully Convolutional Network with Reinforcement Learning for Image Processing](https://mlsanity.com/paper/1912.07190/) (2019)
- [Fully Convolutional Network with Multi-Step Reinforcement Learning for Image Processing](https://mlsanity.com/paper/1811.04323/) (2018)
- [DualDn: Dual-domain Denoising via Differentiable ISP](https://mlsanity.com/paper/2409.18783/) (2024)

## Built on by (most cited first, 18 of 18)

- [Towards RAW Object Detection in Diverse Conditions](https://mlsanity.com/paper/2411.15678/) (2024)
- [Neptune-X: Active X-to-Maritime Generation for Universal Maritime Object Detection](https://mlsanity.com/paper/2509.20745/) (2025)
- [Dark-ISP: Enhancing RAW Image Processing for Low-Light Object Detection](https://mlsanity.com/paper/2509.09183/) (2025)
- [Beyond RGB: Adaptive Parallel Processing for RAW Object Detection](https://mlsanity.com/paper/2503.13163/) (2025)
- [RAWild: Sensor-Agnostic RAW Object Detection via Physics-Guided Curve and Grid Modeling](https://mlsanity.com/paper/2605.05941/) (2026)
- [RAW-Adapter: Adapting Pre-trained Visual Model to Camera RAW Images and A Benchmark](https://mlsanity.com/paper/2503.17027/) (2025)
- [γ-Quant: Towards Learnable Quantization for Low-bit Pattern Recognition](https://mlsanity.com/paper/2509.22448/) (2025)
- [RL-AWB: Deep Reinforcement Learning for Auto White Balance Correction in Low-Light Night-time Scenes](https://mlsanity.com/paper/2601.05249/) (2026)
- [CURVE: CLIP-Utilized Reinforcement Learning for Visual Image Enhancement via Simple Image Processing](https://mlsanity.com/paper/2505.23102/) (2025)
- [POS-ISP: Pipeline Optimization at the Sequence Level for Task-aware ISP](https://mlsanity.com/paper/2604.06938/) (2026)
- [UniISP: A Unified ISP Framework for Both Human and Machine Vision](https://mlsanity.com/paper/2605.07359/) (2026)
- [SemiISP/SemiIE: Semi-Supervised Image Signal Processor and Image Enhancement Leveraging One-to-Many Mapping sRGB-to-RAW](https://mlsanity.com/paper/2504.02345/) (2025)
- [From Cheap to Pro: A Learning-based Adaptive Camera Parameter Network for Professional-Style Imaging](https://mlsanity.com/paper/2510.20550/) (2025)
- [MERIT: Multi-domain Efficient RAW Image Translation](https://mlsanity.com/paper/2603.20836/) (2026)
- [EMOVIS: Emotion-Optimized Image Processing](https://mlsanity.com/paper/2605.03131/) (2026)
- [FreqAdapt: Frequency-Adaptive Processing for RAW Object Detection](https://mlsanity.com/paper/2608.03385/) (2026)
- [Real-Time Scene-Adaptive Tone Mapping for High-Dynamic Range Object Detection](https://mlsanity.com/paper/2608.30400/) (2026)
- [RawVLA: Embodied Neural Image Signal Processor For Robotic Manipulation](https://mlsanity.com/paper/2609.37530/) (2026)

## Abstract (arXiv)

Image Signal Processors (ISPs) convert raw sensor signals into digital images, which significantly influence the image quality and the performance of downstream computer vision tasks. Designing ISP pipeline and tuning ISP parameters are two key steps for building an imaging and vision system. To find optimal ISP configurations, recent works use deep neural networks as a proxy to search for ISP parameters or ISP pipelines. However, these methods are primarily designed to maximize the image quality, which are sub-optimal in the performance of high-level computer vision tasks such as detection, recognition, and tracking. Moreover, after training, the learned ISP pipelines are mostly fixed at the inference time, whose performance degrades in dynamic scenes. To jointly optimize ISP structures and parameters, we propose AdaptiveISP, a task-driven and scene-adaptive ISP. One key observation is that for the majority of input images, only a few processing modules are needed to improve the performance of downstream recognition tasks, and only a few inputs require more processing. Based on this, AdaptiveISP utilizes deep reinforcement learning to automatically generate an optimal ISP pipeline and the associated ISP parameters to maximize the detection performance. Experimental results show that AdaptiveISP not only surpasses the prior state-of-the-art methods for object detection but also dynamically manages the trade-off between detection performance and computational cost, especially suitable for scenes with large dynamic range variations. Project website: https://openimaginglab.github.io/AdaptiveISP/.