# Volumetric Propagation Network: Stereo-LiDAR Fusion for Long-Range Depth Estimation

arXiv: https://arxiv.org/abs/2103.12964 · HTML: https://mlsanity.com/paper/2103.12964/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 35 references that are in the map.
- First posted on arXiv 24 Mar 2021.

## Builds on (most cited first, 27 of 27)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [PointCNN: Convolution On X-Transformed Points](https://mlsanity.com/paper/1801.07791/) (2018)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1812.07179/) (2018)
- [End-to-End Learning of Geometry and Context for Deep Stereo Regression](https://mlsanity.com/paper/1703.04309/) (2017)
- [Virtual KITTI 2](https://mlsanity.com/paper/2001.10773/) (2020)
- [Virtual Worlds as Proxy for Multi-Object Tracking Analysis](https://mlsanity.com/paper/1605.06457/) (2016)
- [Sparsity Invariant CNNs](https://mlsanity.com/paper/1708.06500/) (2017)
- [GA-Net: Guided Aggregation Net for End-to-end Stereo Matching](https://mlsanity.com/paper/1904.06587/) (2019)
- [Non-Local Spatial Propagation Network for Depth Completion](https://mlsanity.com/paper/2007.10042/) (2020)
- [Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving](https://mlsanity.com/paper/1906.06310/) (2019)
- [Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image](https://mlsanity.com/paper/1709.07492/) (2017)
- [Deep Parametric Continuous Convolutional Neural Networks](https://mlsanity.com/paper/2101.06742/) (2021)
- [DPSNet: End-to-end Deep Plane Sweep Stereo](https://mlsanity.com/paper/1905.00538/) (2019)
- [Learning Guided Convolutional Network for Depth Completion](https://mlsanity.com/paper/1908.01238/) (2019)
- [CSPN++: Learning Context and Resource Aware Convolutional Spatial Propagation Networks for Depth Completion](https://mlsanity.com/paper/1911.05377/) (2019)
- [DSGN: Deep Stereo Geometry Network for 3D Object Detection](https://mlsanity.com/paper/2001.03398/) (2020)
- [Interpolated Convolutional Networks for 3D Point Cloud Understanding](https://mlsanity.com/paper/1908.04512/) (2019)
- [Learning Joint 2D-3D Representations for Depth Completion](https://mlsanity.com/paper/2012.12402/) (2020)
- [3D LiDAR and Stereo Fusion using Stereo Matching Network with Conditional Cost Volume Normalization](https://mlsanity.com/paper/1904.02917/) (2019)
- [LiStereo: Generate Dense Depth Maps from LIDAR and Stereo Imagery](https://mlsanity.com/paper/1905.02744/) (2019)
- [Stereo Object Matching Network](https://mlsanity.com/paper/2103.12498/) (2021)

## Built on by (most cited first, 15 of 15)

- [PointMixer: MLP-Mixer for Point Cloud Understanding](https://mlsanity.com/paper/2111.11187/) (2021)
- [VolumeFusion: Deep Depth Fusion for 3D Scene Reconstruction](https://mlsanity.com/paper/2108.08623/) (2021)
- [Revisiting Depth Completion from a Stereo Matching Perspective for Cross-domain Generalization](https://mlsanity.com/paper/2312.09254/) (2023)
- [Deep Point Cloud Reconstruction](https://mlsanity.com/paper/2111.11704/) (2021)
- [DoubleTake: Geometry Guided Depth Estimation](https://mlsanity.com/paper/2406.18387/) (2024)
- [Cross-spectral Gated-RGB Stereo Depth Estimation](https://mlsanity.com/paper/2405.12759/) (2024)
- [Gated Stereo: Joint Depth Estimation from Gated and Wide-Baseline Active Stereo Cues](https://mlsanity.com/paper/2305.12955/) (2023)
- [Stereo-LiDAR Depth Estimation with Deformable Propagation and Learned Disparity-Depth Conversion](https://mlsanity.com/paper/2404.07545/) (2024)
- [Gated Fields: Learning Scene Reconstruction from Gated Videos](https://mlsanity.com/paper/2405.19819/) (2024)
- [Sparse LiDAR Assisted Self-supervised Stereo Disparity Estimation](https://mlsanity.com/paper/2112.15355/) (2021)
- [Non-learning Stereo-aided Depth Completion under Mis-projection via Selective Stereo Matching](https://mlsanity.com/paper/2210.01436/) (2022)
- [Robust and accurate depth estimation by fusing LiDAR and Stereo](https://mlsanity.com/paper/2207.06139/) (2022)
- [Active Stereo in the Wild through Virtual Pattern Projection](https://mlsanity.com/paper/2406.04345/) (2024)
- [A Reinforcement Learning Method for Environments with Stochastic Variables: Post-Decision Proximal Policy Optimization with Dual Critic Networks](https://mlsanity.com/paper/2504.05150/) (2025)
- [Preserving Guidance in Cost-Volume Retrieval under Extreme LiDAR Sparsity in Iterative Stereo](https://mlsanity.com/paper/2507.19738/) (2025)

## Abstract (arXiv)

Stereo-LiDAR fusion is a promising task in that we can utilize two different types of 3D perceptions for practical usage -- dense 3D information (stereo cameras) and highly-accurate sparse point clouds (LiDAR). However, due to their different modalities and structures, the method of aligning sensor data is the key for successful sensor fusion. To this end, we propose a geometry-aware stereo-LiDAR fusion network for long-range depth estimation, called volumetric propagation network. The key idea of our network is to exploit sparse and accurate point clouds as a cue for guiding correspondences of stereo images in a unified 3D volume space. Unlike existing fusion strategies, we directly embed point clouds into the volume, which enables us to propagate valid information into nearby voxels in the volume, and to reduce the uncertainty of correspondences. Thus, it allows us to fuse two different input modalities seamlessly and regress a long-range depth map. Our fusion is further enhanced by a newly proposed feature extraction layer for point clouds guided by images: FusionConv. FusionConv extracts point cloud features that consider both semantic (2D image domain) and geometric (3D domain) relations and aid fusion at the volume. Our network achieves state-of-the-art performance on the KITTI and the Virtual-KITTI datasets among recent stereo-LiDAR fusion methods.