mlsanityData to 7 Oct 2026

Deep Cost Ray Fusion for Sparse Depth Video Completion

arXiv 2409.14935 ↗First posted 23 Sep 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we present a learning-based framework for sparse depth video completion. Given a sparse depth map and a color image at a certain viewpoint, our approach makes a cost volume that is constructed on depth hypothesis planes. To effectively fuse sequential cost volumes of the multiple viewpoints for improved depth completion, we introduce a learning-based cost volume fusion framework, namely RayFusion, that effectively leverages the attention mechanism for each pair of overlapped rays in adjacent cost volumes. As a result of leveraging feature statistics accumulated over time, our proposed framework consistently outperforms or rivals state-of-the-art approaches on diverse indoor and outdoor datasets, including the KITTI Depth Completion benchmark, VOID Depth Completion benchmark, and ScanNetV2 dataset, using much fewer network parameters.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 53 of its 70 references that are in the map. First posted on arXiv 23 Sep 2024.

Builds on

53 of 70 references.

References · 53

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  4. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  5. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
  6. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  7. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 780
  8. 2019Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matchingcited by 414
  9. 2017End-to-End Learning of Geometry and Context for Deep Stereo Regressioncited by 359
  10. 2017Sparsity Invariant CNNscited by 325
  11. 2019Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inferencecited by 252
  12. 2021NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Videocited by 241
38 more
  1. 2020PatchmatchNet: Learned Multi-View Patchmatch Stereocited by 204
  2. 2020Non-Local Spatial Propagation Network for Depth Completioncited by 194
  3. 2017Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Imagecited by 159
  4. 2019Cost Volume Pyramid Based Depth Inference for Multi-View Stereocited by 154
  5. 2019Deep Stereo using Adaptive Thin Volume Representation with Uncertainty Awarenesscited by 151
  6. 2021PENet: Towards Precise and Efficient Image Guided Depth Completioncited by 146
  7. 2018Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Cameracited by 140
  8. 2018DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Imagecited by 129
  9. 2021TransMVSNet: Global Context-aware Multi-view Stereo Network with Transformerscited by 117
  10. 2019DPSNet: End-to-end Deep Plane Sweep Stereocited by 115
  11. 2019Learning Guided Convolutional Network for Depth Completioncited by 111
  12. 2018Deep Depth Completion of a Single RGB-D Imagecited by 110
  13. 2019Depth Completion from Sparse LiDAR Data with Depth-Normal Constraintscited by 108
  14. 2023CompletionFormer: Depth Completion with Convolutions and Vision Transformerscited by 92
  15. 2019Unsupervised Depth Completion from Visual Inertial Odometrycited by 88
  16. 2022SimpleRecon: 3D Reconstruction Without 3D Convolutionscited by 83
  17. 2020Don't Forget The Past: Recurrent Depth Estimation from Monocular Videocited by 71
  18. 2020Learning Joint 2D-3D Representations for Depth Completioncited by 71
  19. 2022Dynamic Spatial Propagation Network for Depth Completioncited by 70
  20. 2019Neural RGB->D Sensing: Depth and Uncertainty from a Video Cameracited by 69
  21. 2020DeepVideoMVS: Multi-View Stereo on Video with Recurrent Spatio-Temporal Fusioncited by 68
  22. 2021Unsupervised Depth Completion with Calibrated Backprojection Layerscited by 61
  23. 2019Dense Depth Posterior (DDP) from Single Image and Sparse Rangecited by 57
  24. 2020Uncertainty-Aware CNNs for Depth Completion: Uncertainty from Beginning to Endcited by 55
  25. 2022MVSTER: Epipolar Transformer for Efficient Multi-View Stereocited by 50
  26. 2023LRRU: Long-short Range Recurrent Updating Networks for Depth Completioncited by 48
  27. 2019Multi-View Stereo by Temporal Nonparametric Fusioncited by 46
  28. 2019Depth Coefficients for Depth Completioncited by 41
  29. 2021Depth Completion with Twin Surface Extrapolation at Occlusion Boundariescited by 41
  30. 2021Learning Topology from Synthetic Data for Unsupervised Depth Completioncited by 38
  31. 2018Geo-Supervised Visual Depth Predictioncited by 33
  32. 2019Indoor Depth Completion with Boundary Consistency and Self-Attentioncited by 29
  33. 2022Monitored Distillation for Positive Congruent Depth Completioncited by 24
  34. 2020Depth Completion via Inductive Fusion of Planar LIDAR and Monocular Cameracited by 22
  35. 2019Deep RGB-D Canonical Correlation Analysis For Sparse Depth Completioncited by 20
  36. 2023RayMVSNet++: Learning Ray-based 1D Implicit Fields for Accurate Multi-View Stereocited by 15
  37. 2016Parse Geometry from a Line: Monocular Depth Estimation with Partial Laser Observationcited by 14
  38. 2020Boosting Monocular Depth Estimation with Lightweight 3D Point Fusioncited by 14

50 most cited of 53 All 53 (JSON)

Built on by

Not cited yet.