# Learned, uncertainty-driven adaptive acquisition for photon-efficient scanning microscopy

arXiv: https://arxiv.org/abs/2310.16102 · HTML: https://mlsanity.com/paper/2310.16102/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 75 references that are in the map.
- First posted on arXiv 24 Oct 2023.

## Builds on (most cited first, 25 of 25)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges](https://mlsanity.com/paper/2011.06225/) (2020)
- [A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification](https://mlsanity.com/paper/2107.07511/) (2021)
- [Simple Baselines for Image Restoration](https://mlsanity.com/paper/2204.04676/) (2022)
- [The Perception-Distortion Tradeoff](https://mlsanity.com/paper/1711.06077/) (2017)
- [Noise2Void - Learning Denoising from Single Noisy Images](https://mlsanity.com/paper/1811.10980/) (2018)
- [Conformal Risk Control](https://mlsanity.com/paper/2208.02814/) (2022)
- [Noise2Self: Blind Denoising by Self-Supervision](https://mlsanity.com/paper/1901.11365/) (2019)
- [On instabilities of deep learning in image reconstruction - Does AI come at a cost?](https://mlsanity.com/paper/1902.05300/) (2019)
- [Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imaging](https://mlsanity.com/paper/2202.05265/) (2022)
- [A Poisson-Gaussian Denoising Dataset with Real Fluorescence Microscopy Images](https://mlsanity.com/paper/1812.10366/) (2018)
- [On hallucinations in tomographic image reconstruction](https://mlsanity.com/paper/2012.00646/) (2020)
- [Handheld Mobile Photography in Very Low Light](https://mlsanity.com/paper/1910.11336/) (2019)
- [How to Trust Your Diffusion Model: A Convex Optimization Approach to Conformal Risk Control](https://mlsanity.com/paper/2302.03791/) (2023)
- [Conformal Decision Theory: Safe Autonomous Decisions from Imperfect Predictions](https://mlsanity.com/paper/2310.05921/) (2023)
- [Dancing under the stars: video denoising in starlight](https://mlsanity.com/paper/2204.04210/) (2022)
- [Reliable deep-learning-based phase imaging with uncertainty quantification](https://mlsanity.com/paper/1901.02038/) (2019)
- [Interpretable Unsupervised Diversity Denoising and Artefact Removal](https://mlsanity.com/paper/2104.01374/) (2021)
- [Stochastic Frequency Masking to Improve Super-Resolution and Denoising Networks](https://mlsanity.com/paper/2003.07119/) (2020)
- [Roadmap on Deep Learning for Microscopy](https://mlsanity.com/paper/2303.03793/) (2023)
- [The Next Best Underwater View](https://mlsanity.com/paper/1512.01789/) (2015)

## Built on by (most cited first, 3 of 3)

- [Early Time Classification with Accumulated Accuracy Gap Control](https://mlsanity.com/paper/2402.00857/) (2024)
- [ShuffleFlow: Scalable Posterior Inference for Bayesian Inverse Imaging](https://mlsanity.com/paper/2606.21099/) (2026)
- [QUTCC: Quantile Uncertainty Training and Conformal Calibration for Imaging Inverse Problems](https://mlsanity.com/paper/2507.14760/) (2025)

## Abstract (arXiv)

Scanning microscopy systems, such as confocal and multiphoton microscopy, are powerful imaging tools for probing deep into biological tissue. However, scanning systems have an inherent trade-off between acquisition time, field of view, phototoxicity, and image quality, often resulting in noisy measurements when fast, large field of view, and/or gentle imaging is needed. Deep learning could be used to denoise noisy microscopy measurements, but these algorithms can be prone to hallucination, which can be disastrous for medical and scientific applications. We propose a method to simultaneously denoise and predict pixel-wise uncertainty for scanning microscopy systems, improving algorithm trustworthiness and providing statistical guarantees for deep learning predictions. Furthermore, we propose to leverage this learned, pixel-wise uncertainty to drive an adaptive acquisition technique that rescans only the most uncertain regions of a sample, saving time and reducing the total light dose to the sample. We demonstrate our method on experimental confocal and multiphoton microscopy systems, showing that our uncertainty maps can pinpoint hallucinations in the deep learned predictions. Finally, with our adaptive acquisition technique, we demonstrate up to 16X reduction in acquisition time and total light dose while successfully recovering fine features in the sample and reducing hallucinations. We are the first to demonstrate distribution-free uncertainty quantification for a denoising task with real experimental data and the first to propose adaptive acquisition based on reconstruction uncertainty.