# LiDAR-guided Stereo Matching with a Spatial Consistency Constraint

arXiv: https://arxiv.org/abs/2202.09953 · HTML: https://mlsanity.com/paper/2202.09953/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 63 references that are in the map.
- First posted on arXiv 21 Feb 2022.

## Builds on (most cited first, 8 of 8)

- [Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review](https://mlsanity.com/paper/2004.05224/) (2020)
- [The Fast Bilateral Solver](https://mlsanity.com/paper/1511.03296/) (2015)
- [Confidence Propagation through CNNs for Guided Sparse Depth Regression](https://mlsanity.com/paper/1811.01791/) (2018)
- [Guided Stereo Matching](https://mlsanity.com/paper/1905.10107/) (2019)
- [A New Stereo Benchmarking Dataset for Satellite Images](https://mlsanity.com/paper/1907.04404/) (2019)
- [Real Time Dense Depth Estimation by Fusing Stereo with Sparse Depth Measurements](https://mlsanity.com/paper/1809.07677/) (2018)
- [High-Resolution Depth Maps Based on TOF-Stereo Fusion](https://mlsanity.com/paper/2107.14688/) (2021)
- [Complete Scene Reconstruction by Merging Images and Laser Scans](https://mlsanity.com/paper/1904.09568/) (2019)

## Built on by (most cited first, 5 of 5)

- [Revisiting Depth Completion from a Stereo Matching Perspective for Cross-domain Generalization](https://mlsanity.com/paper/2312.09254/) (2023)
- [Active Stereo Without Pattern Projector](https://mlsanity.com/paper/2309.12315/) (2023)
- [LiDAR-Event Stereo Fusion with Hallucinations](https://mlsanity.com/paper/2408.04633/) (2024)
- [RGB-D-Fusion: Image Conditioned Depth Diffusion of Humanoid Subjects](https://mlsanity.com/paper/2307.15988/) (2023)
- [Active Stereo in the Wild through Virtual Pattern Projection](https://mlsanity.com/paper/2406.04345/) (2024)

## Abstract (arXiv)

The complementary fusion of light detection and ranging (LiDAR) data and image data is a promising but challenging task for generating high-precision and high-density point clouds. This study proposes an innovative LiDAR-guided stereo matching approach called LiDAR-guided stereo matching (LGSM), which considers the spatial consistency represented by continuous disparity or depth changes in the homogeneous region of an image. The LGSM first detects the homogeneous pixels of each LiDAR projection point based on their color or intensity similarity. Next, we propose a riverbed enhancement function to optimize the cost volume of the LiDAR projection points and their homogeneous pixels to improve the matching robustness. Our formulation expands the constraint scopes of sparse LiDAR projection points with the guidance of image information to optimize the cost volume of pixels as much as possible. We applied LGSM to semi-global matching and AD-Census on both simulated and real datasets. When the percentage of LiDAR points in the simulated datasets was 0.16%, the matching accuracy of our method achieved a subpixel level, while that of the original stereo matching algorithm was 3.4 pixels. The experimental results show that LGSM is suitable for indoor, street, aerial, and satellite image datasets and provides good transferability across semi-global matching and AD-Census. Furthermore, the qualitative and quantitative evaluations demonstrate that LGSM is superior to two state-of-the-art optimizing cost volume methods, especially in reducing mismatches in difficult matching areas and refining the boundaries of objects.