# Robust Single-shot Structured Light 3D Imaging via Neural Feature Decoding

arXiv: https://arxiv.org/abs/2512.14028 · HTML: https://mlsanity.com/paper/2512.14028/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 56 references that are in the map.
- First posted on arXiv 16 Dec 2025.

## Builds on (most cited first, 22 of 22)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [ProcTHOR: Large-Scale Embodied AI Using Procedural Generation](https://mlsanity.com/paper/2206.06994/) (2022)
- [RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching](https://mlsanity.com/paper/2109.07547/) (2021)
- [GA-Net: Guided Aggregation Net for End-to-end Stereo Matching](https://mlsanity.com/paper/1904.06587/) (2019)
- [Group-wise Correlation Stereo Network](https://mlsanity.com/paper/1903.04025/) (2019)
- [AANet: Adaptive Aggregation Network for Efficient Stereo Matching](https://mlsanity.com/paper/2004.09548/) (2020)
- [Iterative Geometry Encoding Volume for Stereo Matching](https://mlsanity.com/paper/2303.06615/) (2023)
- [Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstruction](https://mlsanity.com/paper/1803.03893/) (2018)
- [Prompting Depth Anything for 4K Resolution Accurate Metric Depth Estimation](https://mlsanity.com/paper/2412.14015/) (2024)
- [Neural LiDAR Fields for Novel View Synthesis](https://mlsanity.com/paper/2305.01643/) (2023)
- [MoCha-Stereo: Motif Channel Attention Network for Stereo Matching](https://mlsanity.com/paper/2404.06842/) (2024)
- [Uncertainty Guided Adaptive Warping for Robust and Efficient Stereo Matching](https://mlsanity.com/paper/2307.14071/) (2023)
- [ActiveStereoNet: End-to-End Self-Supervised Learning for Active Stereo Systems](https://mlsanity.com/paper/1807.06009/) (2018)
- [Deep Depth From Focus](https://mlsanity.com/paper/1704.01085/) (2017)
- [Polka Lines: Learning Structured Illumination and Reconstruction for Active Stereo](https://mlsanity.com/paper/2011.13117/) (2020)
- [Consistent Direct Time-of-Flight Video Depth Super-Resolution](https://mlsanity.com/paper/2211.08658/) (2022)
- [Depth Estimation by Combining Binocular Stereo and Monocular Structured-Light](https://mlsanity.com/paper/2203.10493/) (2022)

## Built on by (most cited first, 2 of 2)

- [Physically-guided Image Generation for Multi-Projection Mapping](https://mlsanity.com/paper/2606.22477/) (2026)
- [NSL-SLAM: High-Fidelity Neural Structured-Light Depth for Practical SLAM and Reconstruction](https://mlsanity.com/paper/2607.24495/) (2026)

## Abstract (arXiv)

We consider the problem of active 3D imaging using single-shot structured light systems, which are widely employed in commercial 3D sensing devices such as Apple Face ID and Intel RealSense. Traditional structured light methods typically decode depth correspondences through pixel-domain matching algorithms, resulting in limited robustness under challenging scenarios like occlusions, fine-structured details, and non-Lambertian surfaces. Inspired by recent advances in neural feature matching, we propose a learning-based structured light decoding framework that performs robust correspondence matching within feature space rather than the fragile pixel domain. Our method extracts neural features from the projected patterns and captured infrared (IR) images, explicitly incorporating their geometric priors by building cost volumes in feature space, achieving substantial performance improvements over pixel-domain decoding approaches. To further enhance depth quality, we introduce a depth refinement module that leverages strong priors from large-scale monocular depth estimation models, improving fine detail recovery and global structural coherence. To facilitate effective learning, we develop a physically-based structured light rendering pipeline, generating nearly one million synthetic pattern-image pairs with diverse objects and materials for indoor settings. Experiments demonstrate that our method, trained exclusively on synthetic data with multiple structured light patterns, generalizes well to real-world indoor environments, effectively processes various pattern types without retraining, and consistently outperforms both commercial structured light systems and passive stereo RGB-based depth estimation methods. Project page: https://namisntimpot.github.io/NSLweb/.