# Sketched RT3D: How to reconstruct billions of photons per second

arXiv: https://arxiv.org/abs/2203.00952 · HTML: https://mlsanity.com/paper/2203.00952/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 19 references that are in the map.
- First posted on arXiv 2 Mar 2022.

## Builds on (most cited first, 3 of 3)

- [Real-time 3D reconstruction from single-photon lidar data using plug-and-play point cloud denoisers](https://mlsanity.com/paper/1905.06700/) (2019)
- [A Sketching Framework for Reduced Data Transfer in Photon Counting Lidar](https://mlsanity.com/paper/2102.08732/) (2021)
- [Bayesian 3D Reconstruction of Complex Scenes from Single-Photon Lidar Data](https://mlsanity.com/paper/1810.11633/) (2018)

## Built on by (most cited first, 2 of 2)

- [Single-Photon 3D Imaging with Equi-Depth Photon Histograms](https://mlsanity.com/paper/2408.16150/) (2024)
- [High-Flux Count-Free Single-Photon 3D Cameras](https://mlsanity.com/paper/2608.18306/) (2026)

## Abstract (arXiv)

Single-photon light detection and ranging (lidar) captures depth and intensity information of a 3D scene. Reconstructing a scene from observed photons is a challenging task due to spurious detections associated with background illumination sources. To tackle this problem, there is a plethora of 3D reconstruction algorithms which exploit spatial regularity of natural scenes to provide stable reconstructions. However, most existing algorithms have computational and memory complexity proportional to the number of recorded photons. This complexity hinders their real-time deployment on modern lidar arrays which acquire billions of photons per second. Leveraging a recent lidar sketching framework, we show that it is possible to modify existing reconstruction algorithms such that they only require a small sketch of the photon information. In particular, we propose a sketched version of a recent state-of-the-art algorithm which uses point cloud denoisers to provide spatially regularized reconstructions. A series of experiments performed on real lidar datasets demonstrates a significant reduction of execution time and memory requirements, while achieving the same reconstruction performance than in the full data case.