# Gated2Depth: Real-time Dense Lidar from Gated Images

arXiv: https://arxiv.org/abs/1902.04997 · HTML: https://mlsanity.com/paper/1902.04997/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Feb 2019.

## Built on by (most cited first, 23 of 23)

- [Multi-modal Sensor Fusion for Auto Driving Perception: A Survey](https://mlsanity.com/paper/2202.02703/) (2022)
- [A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlook](https://mlsanity.com/paper/2401.01454/) (2024)
- [Radar-Camera Fusion for Object Detection and Semantic Segmentation in Autonomous Driving: A Comprehensive Review](https://mlsanity.com/paper/2304.10410/) (2023)
- [Emergent Visual Sensors for Autonomous Vehicles](https://mlsanity.com/paper/2205.09383/) (2022)
- [PointNeRF++: A multi-scale, point-based Neural Radiance Field](https://mlsanity.com/paper/2312.02362/) (2023)
- [Gated3D: Monocular 3D Object Detection From Temporal Illumination Cues](https://mlsanity.com/paper/2102.03602/) (2021)
- [Cross-spectral Gated-RGB Stereo Depth Estimation](https://mlsanity.com/paper/2405.12759/) (2024)
- [DurLAR: A High-fidelity 128-channel LiDAR Dataset with Panoramic Ambient and Reflectivity Imagery for Multi-modal Autonomous Driving Applications](https://mlsanity.com/paper/2406.10068/) (2024)
- [Gated Stereo: Joint Depth Estimation from Gated and Wide-Baseline Active Stereo Cues](https://mlsanity.com/paper/2305.12955/) (2023)
- [Pixel-aligned RGB-NIR Stereo Imaging and Dataset for Robot Vision](https://mlsanity.com/paper/2411.18025/) (2024)
- [Gated2Gated: Self-Supervised Depth Estimation from Gated Images](https://mlsanity.com/paper/2112.02416/) (2021)
- [Vulnerable Road User Detection and Safety Enhancement: A Comprehensive Survey](https://mlsanity.com/paper/2405.19202/) (2024)
- [ZeroScatter: Domain Transfer for Long Distance Imaging and Vision through Scattering Media](https://mlsanity.com/paper/2102.05847/) (2021)
- [ISETAuto: Detecting vehicles with depth and radiance information](https://mlsanity.com/paper/2101.01843/) (2021)
- [Learning to See Through Dazzle](https://mlsanity.com/paper/2402.15919/) (2024)
- [Gated Fields: Learning Scene Reconstruction from Gated Videos](https://mlsanity.com/paper/2405.19819/) (2024)
- [SaccadeCam: Adaptive Visual Attention for Monocular Depth Sensing](https://mlsanity.com/paper/2103.12981/) (2021)
- [Refinement of Monocular Depth Maps via Multi-View Differentiable Rendering](https://mlsanity.com/paper/2410.03861/) (2024)
- [ZipDepth: Bringing Lightweight Zero-Shot Monocular Depth Anywhere, on Any Device](https://mlsanity.com/paper/2607.08771/) (2026)
- [Enhancing Lidar-based Object Detection in Adverse Weather using Offset Sequences in Time](https://mlsanity.com/paper/2401.09049/) (2024)
- [Towards Robust Unsupervised Attention Prediction in Autonomous Driving](https://mlsanity.com/paper/2501.15045/) (2025)
- [AIR-VIEW: The Aviation Image Repository for Visibility Estimation of Weather, A Dataset and Benchmark](https://mlsanity.com/paper/2506.20939/) (2025)
- [Towards Autonomous Riding: A Review of Perception, Planning, and Control in Intelligent Two-Wheelers](https://mlsanity.com/paper/2507.11852/) (2025)

## Abstract (arXiv)

We present an imaging framework which converts three images from a gated camera into high-resolution depth maps with depth accuracy comparable to pulsed lidar measurements. Existing scanning lidar systems achieve low spatial resolution at large ranges due to mechanically-limited angular sampling rates, restricting scene understanding tasks to close-range clusters with dense sampling. Moreover, today's pulsed lidar scanners suffer from high cost, power consumption, large form-factors, and they fail in the presence of strong backscatter. We depart from point scanning and demonstrate that it is possible to turn a low-cost CMOS gated imager into a dense depth camera with at least 80m range - by learning depth from three gated images. The proposed architecture exploits semantic context across gated slices, and is trained on a synthetic discriminator loss without the need of dense depth labels. The proposed replacement for scanning lidar systems is real-time, handles back-scatter and provides dense depth at long ranges. We validate our approach in simulation and on real-world data acquired over 4,000km driving in northern Europe. Data and code are available at https://github.com/gruberto/Gated2Depth.