# Adaptive Lighting for Data-Driven Non-Line-of-Sight 3D Localization and Object Identification

arXiv: https://arxiv.org/abs/1905.11595 · HTML: https://mlsanity.com/paper/1905.11595/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2019.

## Built on by (most cited first, 5 of 5)

- [Neural Volumetric Reconstruction for Coherent Synthetic Aperture Sonar](https://mlsanity.com/paper/2306.09909/) (2023)
- [Recent Advances on Non-Line-of-Sight Imaging: Conventional Physical Models, Deep Learning, and New Scenes](https://mlsanity.com/paper/2104.13807/) (2021)
- [What You Can Learn by Staring at a Blank Wall](https://mlsanity.com/paper/2108.13027/) (2021)
- [Few-shot Non-line-of-sight Imaging with Signal-surface Collaborative Regularization](https://mlsanity.com/paper/2211.15367/) (2022)
- [PathFinder: Attention-Driven Dynamic Non-Line-of-Sight Tracking with a Mobile Robot](https://mlsanity.com/paper/2404.05024/) (2024)

## Abstract (arXiv)

Non-line-of-sight (NLOS) imaging of objects not visible to either the camera or illumination source is a challenging task with vital applications including surveillance and robotics. Recent NLOS reconstruction advances have been achieved using time-resolved measurements which requires expensive and specialized detectors and laser sources. In contrast, we propose a data-driven approach for NLOS 3D localization and object identification requiring only a conventional camera and projector. To generalize to complex line-of-sight (LOS) scenes with non-planar surfaces and occlusions, we introduce an adaptive lighting algorithm. This algorithm, based on radiosity, identifies and illuminates scene patches in the LOS which most contribute to the NLOS light paths, and can factor in system power constraints. We achieve an average identification of 87.1% object identification for four classes of objects, and average localization of the NLOS object's centroid with a mean-squared error (MSE) of 1.97 cm in the occluded region for real data taken from a hardware prototype. These results demonstrate the advantage of combining the physics of light transport with active illumination for data-driven NLOS imaging.