Depth on Demand: Streaming Dense Depth from a Low Frame Rate Active Sensor
Not on the map. Why?
Abstract (arXiv)
High frame rate and accurate depth estimation plays an important role in several tasks crucial to robotics and automotive perception. To date, this can be achieved through ToF and LiDAR devices for indoor and outdoor applications, respectively. However, their applicability is limited by low frame rate, energy consumption, and spatial sparsity. Depth on Demand (DoD) allows for accurate temporal and spatial depth densification achieved by exploiting a high frame rate RGB sensor coupled with a potentially lower frame rate and sparse active depth sensor. Our proposal jointly enables lower energy consumption and denser shape reconstruction, by significantly reducing the streaming requirements on the depth sensor thanks to its three core stages: i) multi-modal encoding, ii) iterative multi-modal integration, and iii) depth decoding. We present extended evidence assessing the effectiveness of DoD on indoor and outdoor video datasets, covering both environment scanning and automotive perception use cases.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 42 of its 60 references that are in the map. First posted on arXiv 12 Sep 2024.
Builds on
42 of 60 references.
References · 42
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
- 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
- 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 780
- 2023LightGlue: Local Feature Matching at Light Speedcited by 583
- 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 494
- 2019Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matchingcited by 414
- 2017Sparsity Invariant CNNscited by 325
- 2021NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Videocited by 241
- 2020PatchmatchNet: Learned Multi-View Patchmatch Stereocited by 204
- 2020Non-Local Spatial Propagation Network for Depth Completioncited by 194
- 2019Cost Volume Pyramid Based Depth Inference for Multi-View Stereocited by 154
30 more
- 2019Deep Stereo using Adaptive Thin Volume Representation with Uncertainty Awarenesscited by 151
- 2021PENet: Towards Precise and Efficient Image Guided Depth Completioncited by 146
- 2018Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Cameracited by 140
- 2018Learning Depth with Convolutional Spatial Propagation Networkcited by 135
- 2018Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Networkcited by 109
- 2021TransformerFusion: Monocular RGB Scene Reconstruction using Transformerscited by 107
- 2023CompletionFormer: Depth Completion with Convolutions and Vision Transformerscited by 92
- 2022SimpleRecon: 3D Reconstruction Without 3D Convolutionscited by 83
- 2021RigNet: Repetitive Image Guided Network for Depth Completioncited by 80
- 2022Dynamic Spatial Propagation Network for Depth Completioncited by 70
- 2020Adaptive Context-Aware Multi-Modal Network for Depth Completioncited by 69
- 2024Bilateral Propagation Network for Depth Completioncited by 59
- 2021VolumeFusion: Deep Depth Fusion for 3D Scene Reconstructioncited by 44
- 2024Tri-Perspective View Decomposition for Geometry-Aware Depth Completioncited by 42
- 2019Depth Coefficients for Depth Completioncited by 41
- 2021VoRTX: Volumetric 3D Reconstruction With Transformers for Voxelwise View Selection and Fusioncited by 38
- 2018Propagating Confidences through CNNs for Sparse Data Regressioncited by 36
- 2022Deep Depth Completion from Extremely Sparse Data: A Surveycited by 33
- 2020Project to Adapt: Domain Adaptation for Depth Completion from Noisy and Sparse Sensor Datacited by 28
- 2022DesNet: Decomposed Scale-Consistent Network for Unsupervised Depth Completioncited by 27
- 2022Sparsity Agnostic Depth Completioncited by 27
- 2019Guided Stereo Matchingcited by 24
- 20213DVNet: Multi-View Depth Prediction and Volumetric Refinementcited by 21
- 2021DenseLiDAR: A Real-Time Pseudo Dense Depth Guided Depth Completion Networkcited by 17
- 2022Unsupervised confidence for LiDAR depth maps and applicationscited by 10
- 2018Very Power Efficient Neural Time-of-Flightcited by 8
- 2024RGB Guided ToF Imaging System: A Survey of Deep Learning-based Methodscited by 8
- 2021LiDARTouch: Monocular metric depth estimation with a few-beam LiDARcited by 7
- 2022A Low Memory Footprint Quantized Neural Network for Depth Completion of Very Sparse Time-of-Flight Depth Mapscited by 6
- 2022Multi-View Guided Multi-View Stereocited by 5
Built on by
Citing papers per year: 2024: 1; 2025: 1.
2 papers.