# Real-time Noise Source Estimation of a Camera System from an Image and Metadata

arXiv: https://arxiv.org/abs/2404.03251 · HTML: https://mlsanity.com/paper/2404.03251/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 51 references that are in the map.
- First posted on arXiv 4 Apr 2024.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Benchmarking Denoising Algorithms with Real Photographs](https://mlsanity.com/paper/1707.01313/) (2017)
- [A Physics-based Noise Formation Model for Extreme Low-light Raw Denoising](https://mlsanity.com/paper/2003.12751/) (2020)
- [Noise Flow: Noise Modeling with Conditional Normalizing Flows](https://mlsanity.com/paper/1908.08453/) (2019)
- [Blind2Unblind: Self-Supervised Image Denoising with Visible Blind Spots](https://mlsanity.com/paper/2203.06967/) (2022)
- [Rethinking Noise Synthesis and Modeling in Raw Denoising](https://mlsanity.com/paper/2110.04756/) (2021)
- [Learning Camera-Aware Noise Models](https://mlsanity.com/paper/2008.09370/) (2020)
- [High-level numerical simulations of noise in CCD and CMOS photosensors: review and tutorial](https://mlsanity.com/paper/1412.4031/) (2014)
- [FBI-Denoiser: Fast Blind Image Denoiser for Poisson-Gaussian Noise](https://mlsanity.com/paper/2105.10967/) (2021)
- [Blind estimation of white Gaussian noise variance in highly textured images](https://mlsanity.com/paper/1711.10792/) (2017)
- [Learning-based Noise Component Map Estimation for Image Denoising](https://mlsanity.com/paper/2109.11877/) (2021)
- [Monitoring and Adapting the Physical State of a Camera for Autonomous Vehicles](https://mlsanity.com/paper/2112.05456/) (2021)

## Abstract (arXiv)

Autonomous machines must self-maintain proper functionality to ensure the safety of humans and themselves. This pertains particularly to its cameras as predominant sensors to perceive the environment and support actions. A fundamental camera problem addressed in this study is noise. Solutions often focus on denoising images a posteriori, that is, fighting symptoms rather than root causes. However, tackling root causes requires identifying the noise sources, considering the limitations of mobile platforms. This work investigates a real-time, memory-efficient and reliable noise source estimator that combines data- and physically-based models. To this end, a DNN that examines an image with camera metadata for major camera noise sources is built and trained. In addition, it quantifies unexpected factors that impact image noise or metadata. This study investigates seven different estimators on six datasets that include synthetic noise, real-world noise from two camera systems, and real field campaigns. For these, only the model with most metadata is capable to accurately and robustly quantify all individual noise contributions. This method outperforms total image noise estimators and can be plug-and-play deployed. It also serves as a basis to include more advanced noise sources, or as part of an automatic countermeasure feedback-loop to approach fully reliable machines.