# TransiT: Transient Transformer for Non-line-of-sight Videography

arXiv: https://arxiv.org/abs/2503.11328 · HTML: https://mlsanity.com/paper/2503.11328/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 47 references that are in the map.
- First posted on arXiv 14 Mar 2025.

## Builds on (most cited first, 20 of 20)

- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Virtual Wave Optics for Non-Line-of-Sight Imaging](https://mlsanity.com/paper/1810.07535/) (2018)
- [Advances in Deep Concealed Scene Understanding](https://mlsanity.com/paper/2304.11234/) (2023)
- [Non-line-of-sight Imaging](https://mlsanity.com/paper/2005.08026/) (2020)
- [Non-line-of-Sight Imaging via Neural Transient Fields](https://mlsanity.com/paper/2101.00373/) (2021)
- [Fast Back-Projection for Non-Line of Sight Reconstruction](https://mlsanity.com/paper/1703.02016/) (2017)
- [Deep Non-Line-of-Sight Reconstruction](https://mlsanity.com/paper/2001.09067/) (2020)
- [Spatial-Temporal Transformer for Video Snapshot Compressive Imaging](https://mlsanity.com/paper/2209.01578/) (2022)
- [Real-time Non-line-of-Sight imaging of dynamic scenes](https://mlsanity.com/paper/2010.12737/) (2020)
- [Physics to the Rescue: Deep Non-line-of-sight Reconstruction for High-speed Imaging](https://mlsanity.com/paper/2205.01679/) (2022)
- [Keyhole Imaging: Non-Line-of-Sight Imaging and Tracking of Moving Objects Along a Single Optical Path](https://mlsanity.com/paper/1912.06727/) (2019)
- [Efficient Non-Line-of-Sight Imaging from Transient Sinograms](https://mlsanity.com/paper/2008.02787/) (2020)
- [Recent Advances on Non-Line-of-Sight Imaging: Conventional Physical Models, Deep Learning, and New Scenes](https://mlsanity.com/paper/2104.13807/) (2021)
- [Non-line-of-sight imaging with arbitrary illumination and detection pattern](https://mlsanity.com/paper/2211.00648/) (2022)
- [Few-shot Non-line-of-sight Imaging with Signal-surface Collaborative Regularization](https://mlsanity.com/paper/2211.15367/) (2022)
- [Enhancing Autonomous Navigation by Imaging Hidden Objects using Single-Photon LiDAR](https://mlsanity.com/paper/2410.03555/) (2024)
- [Propagate And Calibrate: Real-time Passive Non-line-of-sight Tracking](https://mlsanity.com/paper/2303.11791/) (2023)

## Abstract (arXiv)

High quality and high speed videography using Non-Line-of-Sight (NLOS) imaging benefit autonomous navigation, collision prevention, and post-disaster search and rescue tasks. Current solutions have to balance between the frame rate and image quality. High frame rates, for example, can be achieved by reducing either per-point scanning time or scanning density, but at the cost of lowering the information density at individual frames. Fast scanning process further reduces the signal-to-noise ratio and different scanning systems exhibit different distortion characteristics. In this work, we design and employ a new Transient Transformer architecture called TransiT to achieve real-time NLOS recovery under fast scans. TransiT directly compresses the temporal dimension of input transients to extract features, reducing computation costs and meeting high frame rate requirements. It further adopts a feature fusion mechanism as well as employs a spatial-temporal Transformer to help capture features of NLOS transient videos. Moreover, TransiT applies transfer learning to bridge the gap between synthetic and real-measured data. In real experiments, TransiT manages to reconstruct from sparse transients of 16 × 16 measured at an exposure time of 0.4 ms per point to NLOS videos at a 64 × 64 resolution at 10 frames per second. We will make our code and dataset available to the community.