# bit2bit: 1-bit quanta video reconstruction via self-supervised photon prediction

arXiv: https://arxiv.org/abs/2410.23247 · HTML: https://mlsanity.com/paper/2410.23247/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 42 references that are in the map.
- First posted on arXiv 30 Oct 2024.

## Builds on (most cited first, 20 of 20)

- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Training Deep Nets with Sublinear Memory Cost](https://mlsanity.com/paper/1604.06174/) (2016)
- [Noise2Noise: Learning Image Restoration without Clean Data](https://mlsanity.com/paper/1803.04189/) (2018)
- [Noise2Void - Learning Denoising from Single Noisy Images](https://mlsanity.com/paper/1811.10980/) (2018)
- [Noise2Self: Blind Denoising by Self-Supervision](https://mlsanity.com/paper/1901.11365/) (2019)
- [ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data](https://mlsanity.com/paper/1904.00592/) (2019)
- [Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images](https://mlsanity.com/paper/2101.02824/) (2021)
- [Noisier2Noise: Learning to Denoise from Unpaired Noisy Data](https://mlsanity.com/paper/1910.11908/) (2019)
- [Efficient Image Super-Resolution Using Pixel Attention](https://mlsanity.com/paper/2010.01073/) (2020)
- [A megapixel time-gated SPAD image sensor for 2D and 3D imaging applications](https://mlsanity.com/paper/1912.12910/) (2019)
- [Noise2Fast: Fast Self-Supervised Single Image Blind Denoising](https://mlsanity.com/paper/2108.10209/) (2021)
- [Quanta Burst Photography](https://mlsanity.com/paper/2006.11840/) (2020)
- [Image Classification in the Dark using Quanta Image Sensors](https://mlsanity.com/paper/2006.02026/) (2020)
- [Motion Adaptive Deblurring with Single-Photon Cameras](https://mlsanity.com/paper/2012.07931/) (2020)
- [What Does a One-Bit Quanta Image Sensor Offer?](https://mlsanity.com/paper/2208.10350/) (2022)
- [Photon Limited Non-Blind Deblurring Using Algorithm Unrolling](https://mlsanity.com/paper/2110.15314/) (2021)
- [N2V2 -- Fixing Noise2Void Checkerboard Artifacts with Modified Sampling Strategies and a Tweaked Network Architecture](https://mlsanity.com/paper/2211.08512/) (2022)
- [Image Denoising and the Generative Accumulation of Photons](https://mlsanity.com/paper/2307.06607/) (2023)

## Built on by (most cited first, 2 of 2)

- [Joint Depth and Reflectivity Estimation using Single-Photon LiDAR](https://mlsanity.com/paper/2505.13250/) (2025)
- [gQIR: Generative Quanta Image Reconstruction](https://mlsanity.com/paper/2602.20417/) (2026)

## Abstract (arXiv)

Quanta image sensors, such as SPAD arrays, are an emerging sensor technology, producing 1-bit arrays representing photon detection events over exposures as short as a few nanoseconds. In practice, raw data are post-processed using heavy spatiotemporal binning to create more useful and interpretable images at the cost of degrading spatiotemporal resolution. In this work, we propose bit2bit, a new method for reconstructing high-quality image stacks at the original spatiotemporal resolution from sparse binary quanta image data. Inspired by recent work on Poisson denoising, we developed an algorithm that creates a dense image sequence from sparse binary photon data by predicting the photon arrival location probability distribution. However, due to the binary nature of the data, we show that the assumption of a Poisson distribution is inadequate. Instead, we model the process with a Bernoulli lattice process from the truncated Poisson. This leads to the proposal of a novel self-supervised solution based on a masked loss function. We evaluate our method using both simulated and real data. On simulated data from a conventional video, we achieve 34.35 mean PSNR with extremely photon-sparse binary input (<0.06 photons per pixel per frame). We also present a novel dataset containing a wide range of real SPAD high-speed videos under various challenging imaging conditions. The scenes cover strong/weak ambient light, strong motion, ultra-fast events, etc., which will be made available to the community, on which we demonstrate the promise of our approach. Both reconstruction quality and throughput substantially surpass the state-of-the-art methods (e.g., Quanta Burst Photography (QBP)). Our approach significantly enhances the visualization and usability of the data, enabling the application of existing analysis techniques.