# Learning Joint 2D-3D Representations for Depth Completion

arXiv: https://arxiv.org/abs/2012.12402 · HTML: https://mlsanity.com/paper/2012.12402/

## Key facts

- Cited by 71 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Dec 2020.

## Built on by (most cited first, 30 of 71)

- [PENet: Towards Precise and Efficient Image Guided Depth Completion](https://mlsanity.com/paper/2103.00783/) (2021)
- [RigNet: Repetitive Image Guided Network for Depth Completion](https://mlsanity.com/paper/2107.13802/) (2021)
- [GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Driving](https://mlsanity.com/paper/2101.06543/) (2021)
- [RGB-D Local Implicit Function for Depth Completion of Transparent Objects](https://mlsanity.com/paper/2104.00622/) (2021)
- [Unsupervised Depth Completion with Calibrated Backprojection Layers](https://mlsanity.com/paper/2108.10531/) (2021)
- [Bilateral Propagation Network for Depth Completion](https://mlsanity.com/paper/2403.11270/) (2024)
- [Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusion](https://mlsanity.com/paper/2412.13389/) (2024)
- [Exploring Adversarial Robustness of Multi-Sensor Perception Systems in Self Driving](https://mlsanity.com/paper/2101.06784/) (2021)
- [Tri-Perspective View Decomposition for Geometry-Aware Depth Completion](https://mlsanity.com/paper/2403.15008/) (2024)
- [Depth Completion with Twin Surface Extrapolation at Occlusion Boundaries](https://mlsanity.com/paper/2104.02253/) (2021)
- [Learning Topology from Synthetic Data for Unsupervised Depth Completion](https://mlsanity.com/paper/2106.02994/) (2021)
- [Deep Depth Completion from Extremely Sparse Data: A Survey](https://mlsanity.com/paper/2205.05335/) (2022)
- [An Adaptive Framework for Learning Unsupervised Depth Completion](https://mlsanity.com/paper/2106.03010/) (2021)
- [DELTAR: Depth Estimation from a Light-weight ToF Sensor and RGB Image](https://mlsanity.com/paper/2209.13362/) (2022)
- [Test-Time Adaptation for Depth Completion](https://mlsanity.com/paper/2402.03312/) (2024)
- [Depth Completion using Plane-Residual Representation](https://mlsanity.com/paper/2104.07350/) (2021)
- [Monitored Distillation for Positive Congruent Depth Completion](https://mlsanity.com/paper/2203.16034/) (2022)
- [Invisible Stitch: Generating Smooth 3D Scenes with Depth Inpainting](https://mlsanity.com/paper/2404.19758/) (2024)
- [Bayesian Deep Basis Fitting for Depth Completion with Uncertainty](https://mlsanity.com/paper/2103.15254/) (2021)
- [SemAttNet: Towards Attention-based Semantic Aware Guided Depth Completion](https://mlsanity.com/paper/2204.13635/) (2022)
- [GraphCSPN: Geometry-Aware Depth Completion via Dynamic GCNs](https://mlsanity.com/paper/2210.10758/) (2022)
- [A Surface Geometry Model for LiDAR Depth Completion](https://mlsanity.com/paper/2104.08466/) (2021)
- [DenseLiDAR: A Real-Time Pseudo Dense Depth Guided Depth Completion Network](https://mlsanity.com/paper/2108.12655/) (2021)
- [Robust Depth Completion with Uncertainty-Driven Loss Functions](https://mlsanity.com/paper/2112.07895/) (2021)
- [Volumetric Propagation Network: Stereo-LiDAR Fusion for Long-Range Depth Estimation](https://mlsanity.com/paper/2103.12964/) (2021)
- [Distilling Monocular Foundation Model for Fine-grained Depth Completion](https://mlsanity.com/paper/2503.16970/) (2025)
- [Advancing Self-supervised Monocular Depth Learning with Sparse LiDAR](https://mlsanity.com/paper/2109.09628/) (2021)
- [TODE-Trans: Transparent Object Depth Estimation with Transformer](https://mlsanity.com/paper/2209.08455/) (2022)
- [Depth Completion using Geometry-Aware Embedding](https://mlsanity.com/paper/2203.10912/) (2022)
- [Semantic Scene Completion with Cleaner Self](https://mlsanity.com/paper/2303.09977/) (2023)

## Abstract (arXiv)

In this paper, we tackle the problem of depth completion from RGBD data. Towards this goal, we design a simple yet effective neural network block that learns to extract joint 2D and 3D features. Specifically, the block consists of two domain-specific sub-networks that apply 2D convolution on image pixels and continuous convolution on 3D points, with their output features fused in image space. We build the depth completion network simply by stacking the proposed block, which has the advantage of learning hierarchical representations that are fully fused between 2D and 3D spaces at multiple levels. We demonstrate the effectiveness of our approach on the challenging KITTI depth completion benchmark and show that our approach outperforms the state-of-the-art.