# Depth Estimation by Combining Binocular Stereo and Monocular Structured-Light

arXiv: https://arxiv.org/abs/2203.10493 · HTML: https://mlsanity.com/paper/2203.10493/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 51 references that are in the map.
- First posted on arXiv 20 Mar 2022.

## Builds on (most cited first, 30 of 30)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching](https://mlsanity.com/paper/1912.06378/) (2019)
- [End-to-End Learning of Geometry and Context for Deep Stereo Regression](https://mlsanity.com/paper/1703.04309/) (2017)
- [GA-Net: Guided Aggregation Net for End-to-end Stereo Matching](https://mlsanity.com/paper/1904.06587/) (2019)
- [Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates](https://mlsanity.com/paper/1708.07120/) (2017)
- [Non-Local Spatial Propagation Network for Depth Completion](https://mlsanity.com/paper/2007.10042/) (2020)
- [Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image](https://mlsanity.com/paper/1709.07492/) (2017)
- [CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching](https://mlsanity.com/paper/2104.04314/) (2021)
- [Deep Stereo using Adaptive Thin Volume Representation with Uncertainty Awareness](https://mlsanity.com/paper/1911.12012/) (2019)
- [DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Image](https://mlsanity.com/paper/1812.00488/) (2018)
- [HITNet: Hierarchical Iterative Tile Refinement Network for Real-time Stereo Matching](https://mlsanity.com/paper/2007.12140/) (2020)
- [DeepPruner: Learning Efficient Stereo Matching via Differentiable PatchMatch](https://mlsanity.com/paper/1909.05845/) (2019)
- [Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network](https://mlsanity.com/paper/1808.00150/) (2018)
- [Depth Completion from Sparse LiDAR Data with Depth-Normal Constraints](https://mlsanity.com/paper/1910.06727/) (2019)
- [SegStereo: Exploiting Semantic Information for Disparity Estimation](https://mlsanity.com/paper/1807.11699/) (2018)
- [Sparse and Dense Data with CNNs: Depth Completion and Semantic Segmentation](https://mlsanity.com/paper/1808.00769/) (2018)
- [Computing the Stereo Matching Cost with a Convolutional Neural Network](https://mlsanity.com/paper/1409.4326/) (2014)
- [Cascade Residual Learning: A Two-stage Convolutional Neural Network for Stereo Matching](https://mlsanity.com/paper/1708.09204/) (2017)
- [Learning Parallax Attention for Stereo Image Super-Resolution](https://mlsanity.com/paper/1903.05784/) (2019)
- [Bilateral Grid Learning for Stereo Matching Networks](https://mlsanity.com/paper/2101.01601/) (2021)
- [Depth Completion with Twin Surface Extrapolation at Occlusion Boundaries](https://mlsanity.com/paper/2104.02253/) (2021)
- [EdgeStereo: An Effective Multi-Task Learning Network for Stereo Matching and Edge Detection](https://mlsanity.com/paper/1903.01700/) (2019)
- [Parallax Attention for Unsupervised Stereo Correspondence Learning](https://mlsanity.com/paper/2009.08250/) (2020)
- [ActiveStereoNet: End-to-End Self-Supervised Learning for Active Stereo Systems](https://mlsanity.com/paper/1807.06009/) (2018)
- [A Decomposition Model for Stereo Matching](https://mlsanity.com/paper/2104.07516/) (2021)
- [Guided Stereo Matching](https://mlsanity.com/paper/1905.10107/) (2019)
- [Kinect Range Sensing: Structured-Light versus Time-of-Flight Kinect](https://mlsanity.com/paper/1505.05459/) (2015)
- [IRS: A Large Naturalistic Indoor Robotics Stereo Dataset to Train Deep Models for Disparity and Surface Normal Estimation](https://mlsanity.com/paper/1912.09678/) (2019)

## Built on by (most cited first, 4 of 4)

- [A Survey on Deep Stereo Matching in the Twenties](https://mlsanity.com/paper/2407.07816/) (2024)
- [Drone-NeRF: Efficient NeRF Based 3D Scene Reconstruction for Large-Scale Drone Survey](https://mlsanity.com/paper/2308.15733/) (2023)
- [Robust Single-shot Structured Light 3D Imaging via Neural Feature Decoding](https://mlsanity.com/paper/2512.14028/) (2025)
- [NSL-SLAM: High-Fidelity Neural Structured-Light Depth for Practical SLAM and Reconstruction](https://mlsanity.com/paper/2607.24495/) (2026)

## Abstract (arXiv)

It is well known that the passive stereo system cannot adapt well to weak texture objects, e.g., white walls. However, these weak texture targets are very common in indoor environments. In this paper, we present a novel stereo system, which consists of two cameras (an RGB camera and an IR camera) and an IR speckle projector. The RGB camera is used both for depth estimation and texture acquisition. The IR camera and the speckle projector can form a monocular structured-light (MSL) subsystem, while the two cameras can form a binocular stereo subsystem. The depth map generated by the MSL subsystem can provide external guidance for the stereo matching networks, which can improve the matching accuracy significantly. In order to verify the effectiveness of the proposed system, we build a prototype and collect a test dataset in indoor scenes. The evaluation results show that the Bad 2.0 error of the proposed system is 28.2% of the passive stereo system when the network RAFT is used. The dataset and trained models are available at https://github.com/YuhuaXu/MonoStereoFusion.