# Preserving Guidance in Cost-Volume Retrieval under Extreme LiDAR Sparsity in Iterative Stereo

arXiv: https://arxiv.org/abs/2507.19738 · HTML: https://mlsanity.com/paper/2507.19738/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 48 of its 65 references that are in the map.
- First posted on arXiv 26 Jul 2025.

## Builds on (most cited first, 30 of 48)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [End-to-End Learning of Geometry and Context for Deep Stereo Regression](https://mlsanity.com/paper/1703.04309/) (2017)
- [Virtual KITTI 2](https://mlsanity.com/paper/2001.10773/) (2020)
- [RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching](https://mlsanity.com/paper/2109.07547/) (2021)
- [Sparsity Invariant CNNs](https://mlsanity.com/paper/1708.06500/) (2017)
- [Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation](https://mlsanity.com/paper/2203.11483/) (2022)
- [Non-Local Spatial Propagation Network for Depth Completion](https://mlsanity.com/paper/2007.10042/) (2020)
- [Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2011.02910/) (2020)
- [Iterative Geometry Encoding Volume for Stereo Matching](https://mlsanity.com/paper/2303.06615/) (2023)
- [Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving](https://mlsanity.com/paper/1906.06310/) (2019)
- [SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flow](https://mlsanity.com/paper/2405.14793/) (2024)
- [FoundationStereo: Zero-Shot Stereo Matching](https://mlsanity.com/paper/2501.09898/) (2025)
- [CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flow](https://mlsanity.com/paper/2211.10408/) (2022)
- [PENet: Towards Precise and Efficient Image Guided Depth Completion](https://mlsanity.com/paper/2103.00783/) (2021)
- [DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Image](https://mlsanity.com/paper/1812.00488/) (2018)
- [Hierarchical Deep Stereo Matching on High-resolution Images](https://mlsanity.com/paper/1912.06704/) (2019)
- [Learning Guided Convolutional Network for Depth Completion](https://mlsanity.com/paper/1908.01238/) (2019)
- [Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network](https://mlsanity.com/paper/1808.00150/) (2018)
- [Selective-Stereo: Adaptive Frequency Information Selection for Stereo Matching](https://mlsanity.com/paper/2403.00486/) (2024)
- [In Defense of Classical Image Processing: Fast Depth Completion on the CPU](https://mlsanity.com/paper/1802.00036/) (2018)
- [SegStereo: Exploiting Semantic Information for Disparity Estimation](https://mlsanity.com/paper/1807.11699/) (2018)
- [Learning Affinity via Spatial Propagation Networks](https://mlsanity.com/paper/1710.01020/) (2017)
- [Accurate and Efficient Stereo Matching via Attention Concatenation Volume](https://mlsanity.com/paper/2209.12699/) (2022)
- [LRRU: Long-short Range Recurrent Updating Networks for Depth Completion](https://mlsanity.com/paper/2310.08956/) (2023)
- [NeRF-Supervised Deep Stereo](https://mlsanity.com/paper/2303.17603/) (2023)
- [FCFR-Net: Feature Fusion based Coarse-to-Fine Residual Learning for Depth Completion](https://mlsanity.com/paper/2012.08270/) (2020)
- [GraftNet: Towards Domain Generalized Stereo Matching with a Broad-Spectrum and Task-Oriented Feature](https://mlsanity.com/paper/2204.00179/) (2022)

## Abstract (arXiv)

While accurate LiDAR depth has been shown to improve stereo matching, high-end LiDAR remains costly and difficult to deploy at scale, motivating guidance from sparse LiDAR measurements. In this paper, we revisit how extremely sparse LiDAR can guide iterative stereo by examining its cost-volume retrieval mechanism. Our analysis shows that LiDAR guidance degrades sharply when only a few hundred points are available, and we provide a signal-processing explanation for why iterative stereo fails to fully exploit such sparse input. This insight leads to a simple yet highly effective remedy: pre-filling the initial disparity map so that cost-volume retrieval becomes more reliable under extremely sparse LiDAR. We find that pre-filling is also effective when injecting LiDAR depth into image features via early fusion, but for a fundamentally different reason, necessitating a distinct pre-filling approach. Combining both forms of guidance, our proposed Guided RAFT-Stereo (GRAFT-Stereo) achieves substantial improvements over existing LiDAR-guided stereo methods across multiple datasets.