# Monitoring and Adapting the Physical State of a Camera for Autonomous Vehicles

arXiv: https://arxiv.org/abs/2112.05456 · HTML: https://mlsanity.com/paper/2112.05456/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 65 references that are in the map.
- First posted on arXiv 10 Dec 2021.

## Builds on (most cited first, 12 of 12)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [A Survey of Deep Learning Applications to Autonomous Vehicle Control](https://mlsanity.com/paper/1912.10773/) (2019)
- [Real-world Noisy Image Denoising: A New Benchmark](https://mlsanity.com/paper/1804.02603/) (2018)
- [ROAD: The ROad event Awareness Dataset for Autonomous Driving](https://mlsanity.com/paper/2102.11585/) (2021)
- [Graph-Based Blind Image Deblurring From a Single Photograph](https://mlsanity.com/paper/1802.07929/) (2018)
- [High-level numerical simulations of noise in CCD and CMOS photosensors: review and tutorial](https://mlsanity.com/paper/1412.4031/) (2014)
- [Camera Exposure Control for Robust Robot Vision with Noise-Aware Image Quality Assessment](https://mlsanity.com/paper/1907.12646/) (2019)
- [Gradient-based Camera Exposure Control for Outdoor Mobile Platforms](https://mlsanity.com/paper/1708.07338/) (2017)
- [Automatic Estimation of Modulation Transfer Functions](https://mlsanity.com/paper/1805.01872/) (2018)
- [Sensor Artificial Intelligence and its Application to Space Systems -- A White Paper](https://mlsanity.com/paper/2006.08368/) (2020)

## Built on by (most cited first, 2 of 2)

- [Real-time Noise Source Estimation of a Camera System from an Image and Metadata](https://mlsanity.com/paper/2404.03251/) (2024)
- [Onboard Out-of-Calibration Detection of Deep Learning Models using Conformal Prediction](https://mlsanity.com/paper/2405.02634/) (2024)

## Abstract (arXiv)

Autonomous vehicles and robots require increasingly more robustness and reliability to meet the demands of modern tasks. These requirements specially apply to cameras onboard such vehicles because they are the predominant sensors to acquire information about the environment and support actions. Cameras must maintain proper functionality and take automatic countermeasures if necessary. Existing solutions are typically tailored to specific problems or detached from the downstream computer vision tasks of the machines, which, however, determine the requirements on the quality of the produced camera images. We propose a generic and task-oriented self-health-maintenance framework for cameras based on data- and physically-grounded models. To this end, we determine two reliable, real-time capable estimators for typical image effects of a camera in poor condition (blur, noise phenomena and most common combinations) by evaluating traditional and customized machine learning-based approaches in extensive experiments. Furthermore, we implement the framework on a real-world ground vehicle and demonstrate how a camera can adjust its parameters to counter an identified poor condition to achieve optimal application capability based on experimental (non-linear and non-monotonic) input-output performance curves. Object detection is chosen as target application, and the image effects motion blur and sensor noise as conditioning examples. Our framework not only provides a practical ready-to-use solution to monitor and maintain the health of cameras, but can also serve as a basis for extensions to tackle more sophisticated problems that combine additional data sources (e.g., sensor or environment parameters) empirically in order to attain fully reliable and robust machines. Code: https://github.com/MaikWischow/Camera-Condition-Monitoring