# Large-scale single-photon imaging

arXiv: https://arxiv.org/abs/2212.13654 · HTML: https://mlsanity.com/paper/2212.13654/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 49 references that are in the map.
- First posted on arXiv 28 Dec 2022.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [A Survey on Visual Transformer](https://mlsanity.com/paper/2012.12556/) (2020)
- [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017)
- [Learning to See in the Dark](https://mlsanity.com/paper/1805.01934/) (2018)
- [Deep Learning for Image Super-resolution: A Survey](https://mlsanity.com/paper/1902.06068/) (2019)
- [A megapixel time-gated SPAD image sensor for 2D and 3D imaging applications](https://mlsanity.com/paper/1912.12910/) (2019)
- [Real-time Non-line-of-Sight imaging of dynamic scenes](https://mlsanity.com/paper/2010.12737/) (2020)
- [AI can evolve without labels: self-evolving vision transformer for chest X-ray diagnosis through knowledge distillation](https://mlsanity.com/paper/2202.06431/) (2022)
- [Large-scale phase retrieval](https://mlsanity.com/paper/2104.03148/) (2021)

## Built on by (most cited first, 2 of 2)

- [Unlocking the Performance of Proximity Sensors by Utilizing Transient Histograms](https://mlsanity.com/paper/2308.13473/) (2023)
- [Streaming quanta sensors for online, high-performance imaging and vision](https://mlsanity.com/paper/2406.00859/) (2024)

## Abstract (arXiv)

Benefiting from its single-photon sensitivity, single-photon avalanche diode (SPAD) array has been widely applied in various fields such as fluorescence lifetime imaging and quantum computing. However, large-scale high-fidelity single-photon imaging remains a big challenge, due to the complex hardware manufacture craft and heavy noise disturbance of SPAD arrays. In this work, we introduce deep learning into SPAD, enabling super-resolution single-photon imaging over an order of magnitude, with significant enhancement of bit depth and imaging quality. We first studied the complex photon flow model of SPAD electronics to accurately characterize multiple physical noise sources, and collected a real SPAD image dataset (64 × 32 pixels, 90 scenes, 10 different bit depth, 3 different illumination flux, 2790 images in total) to calibrate noise model parameters. With this real-world physical noise model, we for the first time synthesized a large-scale realistic single-photon image dataset (image pairs of 5 different resolutions with maximum megapixels, 17250 scenes, 10 different bit depth, 3 different illumination flux, 2.6 million images in total) for subsequent network training. To tackle the severe super-resolution challenge of SPAD inputs with low bit depth, low resolution, and heavy noise, we further built a deep transformer network with a content-adaptive self-attention mechanism and gated fusion modules, which can dig global contextual features to remove multi-source noise and extract full-frequency details. We applied the technique on a series of experiments including macroscopic and microscopic imaging, microfluidic inspection, and Fourier ptychography. The experiments validate the technique's state-of-the-art super-resolution SPAD imaging performance, with more than 5 dB superiority on PSNR compared to the existing methods.