# Photon-Starved Scene Inference using Single Photon Cameras

arXiv: https://arxiv.org/abs/2107.11001 · HTML: https://mlsanity.com/paper/2107.11001/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 45 references that are in the map.
- First posted on arXiv 23 Jul 2021.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [High Quality Monocular Depth Estimation via Transfer Learning](https://mlsanity.com/paper/1812.11941/) (2018)
- [Connecting Image Denoising and High-Level Vision Tasks via Deep Learning](https://mlsanity.com/paper/1809.01826/) (2018)
- [A megapixel time-gated SPAD image sensor for 2D and 3D imaging applications](https://mlsanity.com/paper/1912.12910/) (2019)
- [3D Human Shape and Pose from a Single Low-Resolution Image with Self-Supervised Learning](https://mlsanity.com/paper/2007.13666/) (2020)
- [Asynchronous Single-Photon 3D Imaging](https://mlsanity.com/paper/1908.06372/) (2019)
- [Quanta Burst Photography](https://mlsanity.com/paper/2006.11840/) (2020)
- [High Flux Passive Imaging with Single-Photon Sensors](https://mlsanity.com/paper/1902.10190/) (2019)
- [Photon-Flooded Single-Photon 3D Cameras](https://mlsanity.com/paper/1903.08347/) (2019)
- [Dynamic Low-light Imaging with Quanta Image Sensors](https://mlsanity.com/paper/2007.08614/) (2020)
- [Image Classification in the Dark using Quanta Image Sensors](https://mlsanity.com/paper/2006.02026/) (2020)
- [Seeing into Darkness: Scotopic Visual Recognition](https://mlsanity.com/paper/1610.00405/) (2016)

## Built on by (most cited first, 8 of 8)

- [Streaming quanta sensors for online, high-performance imaging and vision](https://mlsanity.com/paper/2406.00859/) (2024)
- [Radiance Fields from Photons](https://mlsanity.com/paper/2407.09386/) (2024)
- [Robust 3D Object Detection using Probabilistic Point Clouds from Single-Photon LiDARs](https://mlsanity.com/paper/2508.00169/) (2025)
- [Robust Scene Inference under Noise-Blur Dual Corruptions](https://mlsanity.com/paper/2207.11643/) (2022)
- [Enhancing Low-Light Images Using Infrared-Encoded Images](https://mlsanity.com/paper/2307.04122/) (2023)
- [SPC to 3D: Novel View Synthesis from Binary SPC via I2I translation](https://mlsanity.com/paper/2506.06890/) (2025)
- [PhotonSplat: 3D Scene Reconstruction and Colorization from SPAD Sensors](https://mlsanity.com/paper/2506.21680/) (2025)
- [Ultra-low-light computer vision using trained photon correlations](https://mlsanity.com/paper/2604.11993/) (2026)

## Abstract (arXiv)

Scene understanding under low-light conditions is a challenging problem. This is due to the small number of photons captured by the camera and the resulting low signal-to-noise ratio (SNR). Single-photon cameras (SPCs) are an emerging sensing modality that are capable of capturing images with high sensitivity. Despite having minimal read-noise, images captured by SPCs in photon-starved conditions still suffer from strong shot noise, preventing reliable scene inference. We propose photon scale-space a collection of high-SNR images spanning a wide range of photons-per-pixel (PPP) levels (but same scene content) as guides to train inference model on low photon flux images. We develop training techniques that push images with different illumination levels closer to each other in feature representation space. The key idea is that having a spectrum of different brightness levels during training enables effective guidance, and increases robustness to shot noise even in extreme noise cases. Based on the proposed approach, we demonstrate, via simulations and real experiments with a SPAD camera, high-performance on various inference tasks such as image classification and monocular depth estimation under ultra low-light, down to < 1 PPP.