Learning Joint 2D-3D Representations for Depth Completion
Not on the map. Why?
Abstract (arXiv)
In this paper, we tackle the problem of depth completion from RGBD data. Towards this goal, we design a simple yet effective neural network block that learns to extract joint 2D and 3D features. Specifically, the block consists of two domain-specific sub-networks that apply 2D convolution on image pixels and continuous convolution on 3D points, with their output features fused in image space. We build the depth completion network simply by stacking the proposed block, which has the advantage of learning hierarchical representations that are fully fused between 2D and 3D spaces at multiple levels. We demonstrate the effectiveness of our approach on the challenging KITTI depth completion benchmark and show that our approach outperforms the state-of-the-art.
Key facts. Cited by 71 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Dec 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 23; 2022: 15; 2023: 10; 2024: 15; 2025: 7; 2026: 1.
71 papers.
Sort:
By area:
- 2021PENet: Towards Precise and Efficient Image Guided Depth Completioncited by 146
- 2021RigNet: Repetitive Image Guided Network for Depth Completioncited by 80
- 2021GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Drivingcited by 75
- 2021RGB-D Local Implicit Function for Depth Completion of Transparent Objectscited by 64
- 2021Unsupervised Depth Completion with Calibrated Backprojection Layerscited by 61
- 2024Bilateral Propagation Network for Depth Completioncited by 59
- 2024Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusioncited by 47
- 2021Exploring Adversarial Robustness of Multi-Sensor Perception Systems in Self Drivingcited by 45
- 2024Tri-Perspective View Decomposition for Geometry-Aware Depth Completioncited by 42
- 2021Depth Completion with Twin Surface Extrapolation at Occlusion Boundariescited by 41
- 2021Learning Topology from Synthetic Data for Unsupervised Depth Completioncited by 38
- 2022Deep Depth Completion from Extremely Sparse Data: A Surveycited by 33
38 more
- 2021An Adaptive Framework for Learning Unsupervised Depth Completioncited by 30
- 2022DELTAR: Depth Estimation from a Light-weight ToF Sensor and RGB Imagecited by 26
- 2024Test-Time Adaptation for Depth Completioncited by 26
- 2021Depth Completion using Plane-Residual Representationcited by 24
- 2022Monitored Distillation for Positive Congruent Depth Completioncited by 24
- 2021Bayesian Deep Basis Fitting for Depth Completion with Uncertaintycited by 23
- 2022SemAttNet: Towards Attention-based Semantic Aware Guided Depth Completioncited by 23
- 2022GraphCSPN: Geometry-Aware Depth Completion via Dynamic GCNscited by 23
- 2024Invisible Stitch: Generating Smooth 3D Scenes with Depth Inpaintingcited by 23
- 2021A Surface Geometry Model for LiDAR Depth Completioncited by 18
- 2021DenseLiDAR: A Real-Time Pseudo Dense Depth Guided Depth Completion Networkcited by 17
- 2021Robust Depth Completion with Uncertainty-Driven Loss Functionscited by 16
- 2021Volumetric Propagation Network: Stereo-LiDAR Fusion for Long-Range Depth Estimationcited by 15
- 2025Distilling Monocular Foundation Model for Fine-grained Depth Completioncited by 14
- 2021Advancing Self-supervised Monocular Depth Learning with Sparse LiDARcited by 13
- 2022TODE-Trans: Transparent Object Depth Estimation with Transformercited by 13
- 2022Depth Completion using Geometry-Aware Embeddingcited by 12
- 2023Semantic Scene Completion with Cleaner Selfcited by 12
- 2024CaFNet: A Confidence-Driven Framework for Radar Camera Depth Estimationcited by 12
- 2024Completion as Enhancement: A Degradation-Aware Selective Image Guided Network for Depth Completioncited by 12
- 2023RigNet++: Semantic Assisted Repetitive Image Guided Network for Depth Completioncited by 11
- 2021S3: Learnable Sparse Signal Superdensity for Guided Depth Estimationcited by 10
- 2023Decomposed Guided Dynamic Filters for Efficient RGB-Guided Depth Completioncited by 10
- 2023Rapid Network Adaptation: Learning to Adapt Neural Networks Using Test-Time Feedbackcited by 10
- 2023FinnWoodlands Datasetcited by 8
- 2023Active Stereo Without Pattern Projectorcited by 8
- 2023NDDepth: Normal-Distance Assisted Monocular Depth Estimation and Completioncited by 8
- 2024All-day Depth Completioncited by 8
- 2024GET-UP: GEomeTric-aware Depth Estimation with Radar Points UPsamplingcited by 7
- 2022Towards Domain-agnostic Depth Completioncited by 6
- 2022Learning an Efficient Multimodal Depth Completion Modelcited by 6
- 2022SemSegDepth: A Combined Model for Semantic Segmentation and Depth Completioncited by 5
- 2024Scale Propagation Network for Generalizable Depth Completioncited by 5
- 2025Event-Driven Dynamic Scene Depth Completioncited by 5
- 2023Revisiting Deformable Convolution for Depth Completioncited by 4
- 2024DeCoTR: Enhancing Depth Completion with 2D and 3D Attentionscited by 4
- 2025Diffusion Distillation With Direct Preference Optimization For Efficient 3D LiDAR Scene Completioncited by 4
- 2021Similarity-Aware Fusion Network for 3D Semantic Segmentationcited by 3
50 most cited of 71 All 71 (JSON)
- 2026Gaussian Belief Propagation Network for Depth Completion29 Jan 2026
- 2025Training Self-Supervised Depth Completion Using Sparse Measurements and a Single Image20 Jul 2025
- 2025Event-Driven Dynamic Scene Depth Completion19 May 2025
- 2025Depth as Points: Center Point-based Depth Estimation26 Apr 2025
- 2025Diffusion Distillation With Direct Preference Optimization For Efficient 3D LiDAR Scene Completion15 Apr 2025
- 2025GAA-TSO: Geometry-Aware Assisted Depth Completion for Transparent and Specular Objects21 Mar 2025
- 2025Distilling Monocular Foundation Model for Fine-grained Depth Completion21 Mar 2025
- 2025Learning Inverse Laplacian Pyramid for Progressive Depth Completion11 Feb 2025
- 2024Completion as Enhancement: A Degradation-Aware Selective Image Guided Network for Depth Completion26 Dec 2024
- 2024Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusion18 Dec 2024
- 2024Scale Propagation Network for Generalizable Depth Completion24 Oct 2024
- 2024Deep Cost Ray Fusion for Sparse Depth Video Completion23 Sep 2024
38 more
- 2024GET-UP: GEomeTric-aware Depth Estimation with Radar Points UPsampling2 Sep 2024
- 2024VoxDepth: Rectification of Depth Images on Edge Devices21 Jul 2024
- 2024CaFNet: A Confidence-Driven Framework for Radar Camera Depth Estimation30 Jun 2024
- 2024Active Stereo in the Wild through Virtual Pattern Projection6 Jun 2024
- 2024All-day Depth Completion27 May 2024
- 2024Invisible Stitch: Generating Smooth 3D Scenes with Depth Inpainting30 Apr 2024
- 2024Tri-Perspective View Decomposition for Geometry-Aware Depth Completion22 Mar 2024
- 2024DeCoTR: Enhancing Depth Completion with 2D and 3D Attentions18 Mar 2024
- 2024Bilateral Propagation Network for Depth Completion17 Mar 2024
- 2024Test-Time Adaptation for Depth Completion5 Feb 2024
- 2024A Concise but High-performing Network for Image Guided Depth Completion in Autonomous Driving29 Jan 2024
- 2023Self-Supervised Depth Completion Guided by 3D Perception and Geometry Consistency23 Dec 2023
- 2023NDDepth: Normal-Distance Assisted Monocular Depth Estimation and Completion13 Nov 2023
- 2023Gated Cross-Attention Network for Depth Completion28 Sep 2023
- 2023Rapid Network Adaptation: Learning to Adapt Neural Networks Using Test-Time Feedback27 Sep 2023
- 2023Active Stereo Without Pattern Projector21 Sep 2023
- 2023Decomposed Guided Dynamic Filters for Efficient RGB-Guided Depth Completion5 Sep 2023
- 2023RigNet++: Semantic Assisted Repetitive Image Guided Network for Depth Completion1 Sep 2023
- 2023Revisiting Deformable Convolution for Depth Completion3 Aug 2023
- 2023FinnWoodlands Dataset3 Apr 2023
- 2023Semantic Scene Completion with Cleaner Self17 Mar 2023
- 2022PanDepth: Joint Panoptic Segmentation and Depth Completion29 Dec 2022
- 2022GraphCSPN: Geometry-Aware Depth Completion via Dynamic GCNs19 Oct 2022
- 2022Non-learning Stereo-aided Depth Completion under Mis-projection via Selective Stereo Matching4 Oct 2022
- 2022DELTAR: Depth Estimation from a Light-weight ToF Sensor and RGB Image27 Sep 2022
- 2022TODE-Trans: Transparent Object Depth Estimation with Transformer18 Sep 2022
- 2022SemSegDepth: A Combined Model for Semantic Segmentation and Depth Completion1 Sep 2022
- 2022Spacecraft depth completion based on the gray image and the sparse depth map30 Aug 2022
- 2022Learning an Efficient Multimodal Depth Completion Model23 Aug 2022
- 2022Towards Domain-agnostic Depth Completion29 Jul 2022
- 2022How to Guide Adaptive Depth Sampling?20 May 2022
- 2022Deep Depth Completion from Extremely Sparse Data: A Survey11 May 2022
- 2022SemAttNet: Towards Attention-based Semantic Aware Guided Depth Completion28 Apr 2022
- 2022TerrainMesh: Metric-Semantic Terrain Reconstruction from Aerial Images Using Joint 2D-3D Learning23 Apr 2022
- 2022Monitored Distillation for Positive Congruent Depth Completion30 Mar 2022
- 2022Depth Completion using Geometry-Aware Embedding21 Mar 2022
- 2021Robust Depth Completion with Uncertainty-Driven Loss Functions15 Dec 2021
- 2021Advancing Self-supervised Monocular Depth Learning with Sparse LiDAR20 Sep 2021
50 newest of 71 All 71 (JSON)
- 2021PENet: Towards Precise and Efficient Image Guided Depth Completioncited by 146
- 2021RigNet: Repetitive Image Guided Network for Depth Completioncited by 80
- 2021GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Drivingcited by 75
- 2021RGB-D Local Implicit Function for Depth Completion of Transparent Objectscited by 64
- 2021Unsupervised Depth Completion with Calibrated Backprojection Layerscited by 61
- 2024Bilateral Propagation Network for Depth Completioncited by 59
- 2021Exploring Adversarial Robustness of Multi-Sensor Perception Systems in Self Drivingcited by 45
- 2024Tri-Perspective View Decomposition for Geometry-Aware Depth Completioncited by 42
- 2021Depth Completion with Twin Surface Extrapolation at Occlusion Boundariescited by 41
- 2021Learning Topology from Synthetic Data for Unsupervised Depth Completioncited by 38
- 2022Deep Depth Completion from Extremely Sparse Data: A Surveycited by 33
- 2021An Adaptive Framework for Learning Unsupervised Depth Completioncited by 30
12 most cited of 68 All 68 (JSON)
- 2026Gaussian Belief Propagation Network for Depth Completion29 Jan 2026
- 2025Training Self-Supervised Depth Completion Using Sparse Measurements and a Single Image20 Jul 2025
- 2025Event-Driven Dynamic Scene Depth Completion19 May 2025
- 2025Depth as Points: Center Point-based Depth Estimation26 Apr 2025
- 2025GAA-TSO: Geometry-Aware Assisted Depth Completion for Transparent and Specular Objects21 Mar 2025
- 2025Distilling Monocular Foundation Model for Fine-grained Depth Completion21 Mar 2025
- 2025Learning Inverse Laplacian Pyramid for Progressive Depth Completion11 Feb 2025
- 2024Completion as Enhancement: A Degradation-Aware Selective Image Guided Network for Depth Completion26 Dec 2024
- 2024Scale Propagation Network for Generalizable Depth Completion24 Oct 2024
- 2024Deep Cost Ray Fusion for Sparse Depth Video Completion23 Sep 2024
- 2024GET-UP: GEomeTric-aware Depth Estimation with Radar Points UPsampling2 Sep 2024
- 2024VoxDepth: Rectification of Depth Images on Edge Devices21 Jul 2024
12 newest of 68 All 68 (JSON)