# Reconfiguring the Imaging Pipeline for Computer Vision

arXiv: https://arxiv.org/abs/1705.04352 · HTML: https://mlsanity.com/paper/1705.04352/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 May 2017.

## Built on by (most cited first, 30 of 32)

- [Model-Based Image Signal Processors via Learnable Dictionaries](https://mlsanity.com/paper/2201.03210/) (2022)
- [Near-field Perception for Low-Speed Vehicle Automation using Surround-view Fisheye Cameras](https://mlsanity.com/paper/2103.17001/) (2021)
- [DynamicISP: Dynamically Controlled Image Signal Processor for Image Recognition](https://mlsanity.com/paper/2211.01146/) (2022)
- [RAW-Adapter: Adapting Pre-trained Visual Model to Camera RAW Images](https://mlsanity.com/paper/2408.14802/) (2024)
- [Efficient Visual Computing with Camera RAW Snapshots](https://mlsanity.com/paper/2212.07778/) (2022)
- [P2M: A Processing-in-Pixel-in-Memory Paradigm for Resource-Constrained TinyML Applications](https://mlsanity.com/paper/2203.04737/) (2022)
- [Raw or Cooked? Object Detection on RAW Images](https://mlsanity.com/paper/2301.08965/) (2023)
- [RAW-Diffusion: RGB-Guided Diffusion Models for High-Fidelity RAW Image Generation](https://mlsanity.com/paper/2411.13150/) (2024)
- [ISP Distillation](https://mlsanity.com/paper/2101.10203/) (2021)
- [Self-Supervised Reversed Image Signal Processing via Reference-Guided Dynamic Parameter Selection](https://mlsanity.com/paper/2303.13916/) (2023)
- [Raw Bayer Pattern Image Synthesis for Computer Vision-oriented Image Signal Processing Pipeline Design](https://mlsanity.com/paper/2110.12823/) (2021)
- [DRL-ISP: Multi-Objective Camera ISP with Deep Reinforcement Learning](https://mlsanity.com/paper/2207.03081/) (2022)
- [Efficient HDR Reconstruction from Real-World Raw Images](https://mlsanity.com/paper/2306.10311/) (2023)
- [Position: Agentic Systems Constitute a Key Component of Next-Generation Intelligent Image Processing](https://mlsanity.com/paper/2505.16007/) (2025)
- [RAWild: Sensor-Agnostic RAW Object Detection via Physics-Guided Curve and Grid Modeling](https://mlsanity.com/paper/2605.05941/) (2026)
- [Improving the Energy Efficiency and Robustness of tinyML Computer Vision using Log-Gradient Input Images](https://mlsanity.com/paper/2203.02571/) (2022)
- [SimROD: A Simple Baseline for Raw Object Detection with Global and Local Enhancements](https://mlsanity.com/paper/2503.07101/) (2025)
- [RAW-Adapter: Adapting Pre-trained Visual Model to Camera RAW Images and A Benchmark](https://mlsanity.com/paper/2503.17027/) (2025)
- [γ-Quant: Towards Learnable Quantization for Low-bit Pattern Recognition](https://mlsanity.com/paper/2509.22448/) (2025)
- [HeteroSwitch: Characterizing and Taming System-Induced Data Heterogeneity in Federated Learning](https://mlsanity.com/paper/2403.04207/) (2024)
- [RAWDet-7: A Multi-Scenario Benchmark for Object Detection and Description on Quantized RAW Images](https://mlsanity.com/paper/2602.03760/) (2026)
- [UniISP: A Unified ISP Framework for Both Human and Machine Vision](https://mlsanity.com/paper/2605.07359/) (2026)
- [Exploring Data Pipelines through the Process Lens: a Reference Model forComputer Vision](https://mlsanity.com/paper/2107.01824/) (2021)
- [Adaptive Subsampling for ROI-based Visual Tracking: Algorithms and FPGA Implementation](https://mlsanity.com/paper/2112.09775/) (2021)
- [Virtuoso: Video-based Intelligence for real-time tuning on SOCs](https://mlsanity.com/paper/2112.13076/) (2021)
- [Comprehensive Analysis of the Object Detection Pipeline on UAVs](https://mlsanity.com/paper/2203.00306/) (2022)
- [How to deal with glare for improved perception of Autonomous Vehicles](https://mlsanity.com/paper/2404.10992/) (2024)
- [Spatial-Frequency Aware for Object Detection in RAW Image](https://mlsanity.com/paper/2508.01396/) (2025)
- [Learning to Sense for Driving: Joint Optics-Sensor-Model Co-Design for Semantic Segmentation](https://mlsanity.com/paper/2512.20815/) (2025)
- [Saliency Driven Imagery Preprocessing for Efficient Compression -- Industrial Paper](https://mlsanity.com/paper/2601.17555/) (2026)

## Abstract (arXiv)

Advancements in deep learning have ignited an explosion of research on efficient hardware for embedded computer vision. Hardware vision acceleration, however, does not address the cost of capturing and processing the image data that feeds these algorithms. We examine the role of the image signal processing (ISP) pipeline in computer vision to identify opportunities to reduce computation and save energy. The key insight is that imaging pipelines should be designed to be configurable: to switch between a traditional photography mode and a low-power vision mode that produces lower-quality image data suitable only for computer vision. We use eight computer vision algorithms and a reversible pipeline simulation tool to study the imaging system's impact on vision performance. For both CNN-based and classical vision algorithms, we observe that only two ISP stages, demosaicing and gamma compression, are critical for task performance. We propose a new image sensor design that can compensate for skipping these stages. The sensor design features an adjustable resolution and tunable analog-to-digital converters (ADCs). Our proposed imaging system's vision mode disables the ISP entirely and configures the sensor to produce subsampled, lower-precision image data. This vision mode can save ~75% of the average energy of a baseline photography mode while having only a small impact on vision task accuracy.