# Event Guided Depth Sensing

arXiv: https://arxiv.org/abs/2110.10505 · HTML: https://mlsanity.com/paper/2110.10505/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 33 references that are in the map.
- First posted on arXiv 20 Oct 2021.

## Builds on (most cited first, 15 of 15)

- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age](https://mlsanity.com/paper/1606.05830/) (2016)
- [High Speed and High Dynamic Range Video with an Event Camera](https://mlsanity.com/paper/1906.07165/) (2019)
- [Sparsity Invariant CNNs](https://mlsanity.com/paper/1708.06500/) (2017)
- [Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion](https://mlsanity.com/paper/1812.08156/) (2018)
- [End-to-End Learning of Representations for Asynchronous Event-Based Data](https://mlsanity.com/paper/1904.08245/) (2019)
- [Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network](https://mlsanity.com/paper/1808.00150/) (2018)
- [Combining Events and Frames using Recurrent Asynchronous Multimodal Networks for Monocular Depth Prediction](https://mlsanity.com/paper/2102.09320/) (2021)
- [Learning Monocular Dense Depth from Events](https://mlsanity.com/paper/2010.08350/) (2020)
- [Event-Based Motion Segmentation by Motion Compensation](https://mlsanity.com/paper/1904.01293/) (2019)
- [Uncertainty-Aware CNNs for Depth Completion: Uncertainty from Beginning to End](https://mlsanity.com/paper/2006.03349/) (2020)
- [How to Calibrate Your Event Camera](https://mlsanity.com/paper/2105.12362/) (2021)
- [Sparse Depth Sensing for Resource-Constrained Robots](https://mlsanity.com/paper/1703.01398/) (2017)

## Built on by (most cited first, 14 of 14)

- [ESS: Learning Event-based Semantic Segmentation from Still Images](https://mlsanity.com/paper/2203.10016/) (2022)
- [BlinkFlow: A Dataset to Push the Limits of Event-based Optical Flow Estimation](https://mlsanity.com/paper/2303.07716/) (2023)
- [Are High-Resolution Event Cameras Really Needed?](https://mlsanity.com/paper/2203.14672/) (2022)
- [Microsaccade-inspired Event Camera for Robotics](https://mlsanity.com/paper/2405.17769/) (2024)
- [Event-based Shape from Polarization](https://mlsanity.com/paper/2301.06855/) (2023)
- [Unsupervised Domain Adaptation for Training Event-Based Networks Using Contrastive Learning and Uncorrelated Conditioning](https://mlsanity.com/paper/2303.12424/) (2023)
- [Data-Driven Feature Tracking for Event Cameras With and Without Frames](https://mlsanity.com/paper/2211.12826/) (2022)
- [Real-time event simulation with frame-based cameras](https://mlsanity.com/paper/2209.04634/) (2022)
- [Event-Driven Dynamic Scene Depth Completion](https://mlsanity.com/paper/2505.13279/) (2025)
- [X-maps: Direct Depth Lookup for Event-based Structured Light Systems](https://mlsanity.com/paper/2402.10061/) (2024)
- [Event Camera and LiDAR based Human Tracking for Adverse Lighting Conditions in Subterranean Environments](https://mlsanity.com/paper/2304.08908/) (2023)
- [Event Spectroscopy: Event-based Multispectral and Depth Sensing using Structured Light](https://mlsanity.com/paper/2509.06741/) (2025)
- [Structured light with a million light planes per second](https://mlsanity.com/paper/2411.18597/) (2024)
- [Event-based Photometric Stereo via Rotating Illumination and Per-Pixel Learning](https://mlsanity.com/paper/2603.10748/) (2026)

## Abstract (arXiv)

Active depth sensors like structured light, lidar, and time-of-flight systems sample the depth of the entire scene uniformly at a fixed scan rate. This leads to limited spatio-temporal resolution where redundant static information is over-sampled and precious motion information might be under-sampled. In this paper, we present an efficient bio-inspired event-camera-driven depth estimation algorithm. In our approach, we dynamically illuminate areas of interest densely, depending on the scene activity detected by the event camera, and sparsely illuminate areas in the field of view with no motion. The depth estimation is achieved by an event-based structured light system consisting of a laser point projector coupled with a second event-based sensor tuned to detect the reflection of the laser from the scene. We show the feasibility of our approach in a simulated autonomous driving scenario and real indoor sequences using our prototype. We show that, in natural scenes like autonomous driving and indoor environments, moving edges correspond to less than 10% of the scene on average. Thus our setup requires the sensor to scan only 10% of the scene, which could lead to almost 90% less power consumption by the illumination source. While we present the evaluation and proof-of-concept for an event-based structured-light system, the ideas presented here are applicable for a wide range of depth-sensing modalities like LIDAR, time-of-flight, and standard stereo. Video is available at urlhttps://youtu.be/Rvv9IQLYjCQ.