# Flash Photography for Data-Driven Hidden Scene Recovery

arXiv: https://arxiv.org/abs/1810.11710 · HTML: https://mlsanity.com/paper/1810.11710/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Oct 2018.

## Built on by (most cited first, 7 of 7)

- [Recent Advances on Non-Line-of-Sight Imaging: Conventional Physical Models, Deep Learning, and New Scenes](https://mlsanity.com/paper/2104.13807/) (2021)
- [What You Can Learn by Staring at a Blank Wall](https://mlsanity.com/paper/2108.13027/) (2021)
- [Role of Transients in Two-Bounce Non-Line-of-Sight Imaging](https://mlsanity.com/paper/2304.01308/) (2023)
- [Propagate And Calibrate: Real-time Passive Non-line-of-sight Tracking](https://mlsanity.com/paper/2303.11791/) (2023)
- [Automatic calibration of time of flight based non-line-of-sight reconstruction](https://mlsanity.com/paper/2105.10603/) (2021)
- [Passive Non-Line-of-Sight Imaging with Light Transport Modulation](https://mlsanity.com/paper/2312.16014/) (2023)
- [PathFinder: Attention-Driven Dynamic Non-Line-of-Sight Tracking with a Mobile Robot](https://mlsanity.com/paper/2404.05024/) (2024)

## Abstract (arXiv)

Vehicles, search and rescue personnel, and endoscopes use flash lights to locate, identify, and view objects in their surroundings. Here we show the first steps of how all these tasks can be done around corners with consumer cameras. Recent techniques for NLOS imaging using consumer cameras have not been able to both localize and identify the hidden object. We introduce a method that couples traditional geometric understanding and data-driven techniques. To avoid the limitation of large dataset gathering, we train the data-driven models on rendered samples to computationally recover the hidden scene on real data. The method has three independent operating modes: 1) a regression output to localize a hidden object in 2D, 2) an identification output to identify the object type or pose, and 3) a generative network to reconstruct the hidden scene from a new viewpoint. The method is able to localize 12cm wide hidden objects in 2D with 1.7cm accuracy. The method also identifies the hidden object class with 87.7% accuracy (compared to 33.3% random accuracy). This paper also provides an analysis on the distribution of information that encodes the occluded object in the accessible scene. We show that, unlike previously thought, the area that extends beyond the corner is essential for accurate object localization and identification.