# Iterating the Transient Light Transport Matrix for Non-Line-of-Sight Imaging

arXiv: https://arxiv.org/abs/2412.10300 · HTML: https://mlsanity.com/paper/2412.10300/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 63 references that are in the map.
- First posted on arXiv 13 Dec 2024.

## Builds on (most cited first, 10 of 10)

- [Virtual Wave Optics for Non-Line-of-Sight Imaging](https://mlsanity.com/paper/1810.07535/) (2018)
- [Non-line-of-sight Imaging](https://mlsanity.com/paper/2005.08026/) (2020)
- [Real-time Non-line-of-Sight imaging of dynamic scenes](https://mlsanity.com/paper/2010.12737/) (2020)
- [Recent Advances in Imaging Around Corners](https://mlsanity.com/paper/1910.05613/) (2019)
- [Recent Advances on Non-Line-of-Sight Imaging: Conventional Physical Models, Deep Learning, and New Scenes](https://mlsanity.com/paper/2104.13807/) (2021)
- [Non-line-of-sight imaging with arbitrary illumination and detection pattern](https://mlsanity.com/paper/2211.00648/) (2022)
- [Flying with Photons: Rendering Novel Views of Propagating Light](https://mlsanity.com/paper/2404.06493/) (2024)
- [Virtual Light Transport Matrices for Non-Line-Of-Sight Imaging](https://mlsanity.com/paper/2103.12622/) (2021)
- [Virtual Mirrors: Non-Line-of-Sight Imaging Beyond the Third Bounce](https://mlsanity.com/paper/2307.14341/) (2023)
- [Learning to Enhance Aperture Phasor Field for Non-Line-of-Sight Imaging](https://mlsanity.com/paper/2407.18574/) (2024)

## Abstract (arXiv)

Active imaging systems sample the Transient Light Transport Matrix (TLTM) for a scene by sequentially illuminating various positions in this scene using a controllable light source, and then measuring the resulting spatiotemporal light transport with time of flight (ToF) sensors. Time-resolved Non-line-of-sight (NLOS) imaging employs an active imaging system that measures part of the TLTM of an intermediary relay surface, and uses the indirect reflections of light encoded within this TLTM to "see around corners". Such imaging systems have applications in diverse areas such as disaster response, remote surveillance, and autonomous navigation. While existing NLOS imaging systems usually measure a subset of the full TLTM, development of customized gated Single Photon Avalanche Diode (SPAD) arrays citericcardofast-gated2022 has made it feasible to probe the full measurement space. In this work, we demonstrate that the full TLTM on the relay surface can be processed with efficient algorithms to computationally focus and detect our illumination in different parts of the hidden scene, turning the relay surface into a second-order active imaging system. These algorithms allow us to iterate on the measured, first-order TLTM, and extract a second order TLTM for surfaces in the hidden scene. We showcase three applications of TLTMs in NLOS imaging: (1) Scene Relighting with novel illumination, (2) Separation of direct and indirect components of light transport in the hidden scene, and (3) Dual Photography. Additionally, we empirically demonstrate that SPAD arrays enable parallel acquisition of photons, effectively mitigating long acquisition times.