# High-speed object detection with a single-photon time-of-flight image sensor

arXiv: https://arxiv.org/abs/2107.13407 · HTML: https://mlsanity.com/paper/2107.13407/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 45 references that are in the map.
- First posted on arXiv 28 Jul 2021.

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [Multi-View 3D Object Detection Network for Autonomous Driving](https://mlsanity.com/paper/1611.07759/) (2016)
- [3DSSD: Point-based 3D Single Stage Object Detector](https://mlsanity.com/paper/2002.10187/) (2020)
- [A Novel Focal Tversky loss function with improved Attention U-Net for lesion segmentation](https://mlsanity.com/paper/1810.07842/) (2018)
- [Lidar for Autonomous Driving: The principles, challenges, and trends for automotive lidar and perception systems](https://mlsanity.com/paper/2004.08467/) (2020)
- [SuMa++: Efficient LiDAR-based Semantic SLAM](https://mlsanity.com/paper/2105.11320/) (2021)
- [Asynchronous Single-Photon 3D Imaging](https://mlsanity.com/paper/1908.06372/) (2019)
- [An Overview of Depth Cameras and Range Scanners Based on Time-of-Flight Technologies](https://mlsanity.com/paper/2012.06772/) (2020)
- [DeepToF: Off-the-Shelf Real-Time Correction of Multipath Interference in Time-of-Flight Imaging](https://mlsanity.com/paper/1805.09305/) (2018)
- [Single-photon imaging over 200 km](https://mlsanity.com/paper/2103.05860/) (2021)
- [Spatial images from temporal data](https://mlsanity.com/paper/1912.01413/) (2019)
- [Robust super-resolution depth imaging via a multi-feature fusion deep network](https://mlsanity.com/paper/2011.11444/) (2020)
- [Event-based Processing of Single Photon Avalanche Diode Sensors](https://mlsanity.com/paper/2001.02060/) (2019)
- [RGB-D image-based Object Detection: from Traditional Methods to Deep Learning Techniques](https://mlsanity.com/paper/1907.09236/) (2019)

## Built on by (most cited first, 7 of 7)

- [Video super-resolution for single-photon LIDAR](https://mlsanity.com/paper/2210.10474/) (2022)
- [Resolution Limit of Single-Photon LiDAR](https://mlsanity.com/paper/2403.17719/) (2024)
- [DronePose: The identification, segmentation, and orientation detection of drones via neural networks](https://mlsanity.com/paper/2112.05488/) (2021)
- [Long-Distance Field Demonstration of Imaging-Free Drone Identification in Intracity Environments](https://mlsanity.com/paper/2504.20097/) (2025)
- [Joint Depth and Reflectivity Estimation using Single-Photon LiDAR](https://mlsanity.com/paper/2505.13250/) (2025)
- [Semantic Temporal Single-photon LiDAR](https://mlsanity.com/paper/2512.06008/) (2025)
- [SP-TransientBench: A Real-Captured Single Photon Perception Benchmark](https://mlsanity.com/paper/2606.18952/) (2026)

## Abstract (arXiv)

3D time-of-flight (ToF) imaging is used in a variety of applications such as augmented reality (AR), computer interfaces, robotics and autonomous systems. Single-photon avalanche diodes (SPADs) are one of the enabling technologies providing accurate depth data even over long ranges. By developing SPADs in array format with integrated processing combined with pulsed, flood-type illumination, high-speed 3D capture is possible. However, array sizes tend to be relatively small, limiting the lateral resolution of the resulting depth maps, and, consequently, the information that can be extracted from the image for applications such as object detection. In this paper, we demonstrate that these limitations can be overcome through the use of convolutional neural networks (CNNs) for high-performance object detection. We present outdoor results from a portable SPAD camera system that outputs 16-bin photon timing histograms with 64x32 spatial resolution. The results, obtained with exposure times down to 2 ms (equivalent to 500 FPS) and in signal-to-background (SBR) ratios as low as 0.05, point to the advantages of providing the CNN with full histogram data rather than point clouds alone. Alternatively, a combination of point cloud and active intensity data may be used as input, for a similar level of performance. In either case, the GPU-accelerated processing time is less than 1 ms per frame, leading to an overall latency (image acquisition plus processing) in the millisecond range, making the results relevant for safety-critical computer vision applications which would benefit from faster than human reaction times.