# 3D Reconstruction from Transient Measurements with Time-Resolved Transformer

arXiv: https://arxiv.org/abs/2510.09205 · HTML: https://mlsanity.com/paper/2510.09205/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 79 references that are in the map.
- First posted on arXiv 10 Oct 2025.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [Transformer in Transformer](https://mlsanity.com/paper/2103.00112/) (2021)
- [Learning Texture Transformer Network for Image Super-Resolution](https://mlsanity.com/paper/2006.04139/) (2020)
- [GMFlow: Learning Optical Flow via Global Matching](https://mlsanity.com/paper/2111.13680/) (2021)
- [Video Action Transformer Network](https://mlsanity.com/paper/1812.02707/) (2018)
- [ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformer](https://mlsanity.com/paper/2208.14201/) (2022)
- [Non-Local Recurrent Network for Image Restoration](https://mlsanity.com/paper/1806.02919/) (2018)
- [TubeDETR: Spatio-Temporal Video Grounding with Transformers](https://mlsanity.com/paper/2203.16434/) (2022)
- [REGTR: End-to-end Point Cloud Correspondences with Transformers](https://mlsanity.com/paper/2203.14517/) (2022)
- [Virtual Wave Optics for Non-Line-of-Sight Imaging](https://mlsanity.com/paper/1810.07535/) (2018)
- [Dynamic Attentive Graph Learning for Image Restoration](https://mlsanity.com/paper/2109.06620/) (2021)
- [Non-line-of-Sight Imaging via Neural Transient Fields](https://mlsanity.com/paper/2101.00373/) (2021)
- [High Flux Passive Imaging with Single-Photon Sensors](https://mlsanity.com/paper/1902.10190/) (2019)
- [Photon-Flooded Single-Photon 3D Cameras](https://mlsanity.com/paper/1903.08347/) (2019)
- [Spatial-Temporal Transformer for 3D Point Cloud Sequences](https://mlsanity.com/paper/2110.09783/) (2021)
- [Deep Non-Line-of-Sight Reconstruction](https://mlsanity.com/paper/2001.09067/) (2020)
- [Physics to the Rescue: Deep Non-line-of-sight Reconstruction for High-speed Imaging](https://mlsanity.com/paper/2205.01679/) (2022)
- [ChiTransformer:Towards Reliable Stereo from Cues](https://mlsanity.com/paper/2203.04554/) (2022)
- [Single-photon imaging over 200 km](https://mlsanity.com/paper/2103.05860/) (2021)
- [Single-photon computational 3D imaging at 45 km](https://mlsanity.com/paper/1904.10341/) (2019)
- [Adaptive Gating for Single-Photon 3D Imaging](https://mlsanity.com/paper/2111.15047/) (2021)
- [Nonlocal Patch-Based Fully-Connected Tensor Network Decomposition for Remote Sensing Image Inpainting](https://mlsanity.com/paper/2109.05889/) (2021)
- [Super-resolution single-photon imaging at 8.2 kilometers](https://mlsanity.com/paper/2001.11450/) (2020)

## Abstract (arXiv)

Transient measurements, captured by the timeresolved systems, are widely employed in photon-efficient reconstruction tasks, including line-of-sight (LOS) and non-line-of-sight (NLOS) imaging. However, challenges persist in their 3D reconstruction due to the low quantum efficiency of sensors and the high noise levels, particularly for long-range or complex scenes. To boost the 3D reconstruction performance in photon-efficient imaging, we propose a generic Time-Resolved Transformer (TRT) architecture. Different from existing transformers designed for high-dimensional data, TRT has two elaborate attention designs tailored for the spatio-temporal transient measurements. Specifically, the spatio-temporal self-attention encoders explore both local and global correlations within transient data by splitting or downsampling input features into different scales. Then, the spatio-temporal cross attention decoders integrate the local and global features in the token space, resulting in deep features with high representation capabilities. Building on TRT, we develop two task-specific embodiments: TRT-LOS for LOS imaging and TRT-NLOS for NLOS imaging. Extensive experiments demonstrate that both embodiments significantly outperform existing methods on synthetic data and real-world data captured by different imaging systems. In addition, we contribute a large-scale, high-resolution synthetic LOS dataset with various noise levels and capture a set of real-world NLOS measurements using a custom-built imaging system, enhancing the data diversity in this field. Code and datasets are available at https://github.com/Depth2World/TRT.