# Deep RGB-D Canonical Correlation Analysis For Sparse Depth Completion

arXiv: https://arxiv.org/abs/1906.08967 · HTML: https://mlsanity.com/paper/1906.08967/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jun 2019.

## Built on by (most cited first, 20 of 20)

- [CompletionFormer: Depth Completion with Convolutions and Vision Transformers](https://mlsanity.com/paper/2304.13030/) (2023)
- [Toward Practical Monocular Indoor Depth Estimation](https://mlsanity.com/paper/2112.02306/) (2021)
- [Deep Depth Completion from Extremely Sparse Data: A Survey](https://mlsanity.com/paper/2205.05335/) (2022)
- [Synergy between 3DMM and 3D Landmarks for Accurate 3D Facial Geometry](https://mlsanity.com/paper/2110.09772/) (2021)
- [DesNet: Decomposed Scale-Consistent Network for Unsupervised Depth Completion](https://mlsanity.com/paper/2211.10994/) (2022)
- [DenseLiDAR: A Real-Time Pseudo Dense Depth Guided Depth Completion Network](https://mlsanity.com/paper/2108.12655/) (2021)
- [S3: Learnable Sparse Signal Superdensity for Guided Depth Estimation](https://mlsanity.com/paper/2103.02396/) (2021)
- [NDDepth: Normal-Distance Assisted Monocular Depth Estimation and Completion](https://mlsanity.com/paper/2311.07166/) (2023)
- [Learning an Efficient Multimodal Depth Completion Model](https://mlsanity.com/paper/2208.10771/) (2022)
- [RDFC-GAN: RGB-Depth Fusion CycleGAN for Indoor Depth Completion](https://mlsanity.com/paper/2306.03584/) (2023)
- [Sparse SPN: Depth Completion from Sparse Keypoints](https://mlsanity.com/paper/2212.00987/) (2022)
- [Meta-Optimization for Higher Model Generalizability in Single-Image Depth Prediction](https://mlsanity.com/paper/2305.07269/) (2023)
- [InSpaceType: Reconsider Space Type in Indoor Monocular Depth Estimation](https://mlsanity.com/paper/2309.13516/) (2023)
- [Progressive Depth Decoupling and Modulating for Flexible Depth Completion](https://mlsanity.com/paper/2405.09342/) (2024)
- [Transparent Object Depth Completion](https://mlsanity.com/paper/2405.15299/) (2024)
- [Deep Cost Ray Fusion for Sparse Depth Video Completion](https://mlsanity.com/paper/2409.14935/) (2024)
- [Sparse Depth Completion with Semantic Mesh Deformation Optimization](https://mlsanity.com/paper/2112.05498/) (2021)
- [A Multimodal Learning Framework for Comprehensive 3D Mineral Prospectivity Modeling with Jointly Learned Structure-Fluid Relationships](https://mlsanity.com/paper/2309.02911/) (2023)
- [Mask-adaptive Gated Convolution and Bi-directional Progressive Fusion Network for Depth Completion](https://mlsanity.com/paper/2401.07439/) (2024)
- [S2ML: Spatio-Spectral Mutual Learning for Depth Completion](https://mlsanity.com/paper/2511.06033/) (2025)

## Abstract (arXiv)

In this paper, we propose our Correlation For Completion Network (CFCNet), an end-to-end deep learning model that uses the correlation between two data sources to perform sparse depth completion. CFCNet learns to capture, to the largest extent, the semantically correlated features between RGB and depth information. Through pairs of image pixels and the visible measurements in a sparse depth map, CFCNet facilitates feature-level mutual transformation of different data sources. Such a transformation enables CFCNet to predict features and reconstruct data of missing depth measurements according to their corresponding, transformed RGB features. We extend canonical correlation analysis to a 2D domain and formulate it as one of our training objectives (i.e. 2d deep canonical correlation, or "2D2CCA loss"). Extensive experiments validate the ability and flexibility of our CFCNet compared to the state-of-the-art methods on both indoor and outdoor scenes with different real-life sparse patterns. Codes are available at: https://github.com/choyingw/CFCNet.