# An Adaptive Framework for Learning Unsupervised Depth Completion

arXiv: https://arxiv.org/abs/2106.03010 · HTML: https://mlsanity.com/paper/2106.03010/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 49 references that are in the map.
- First posted on arXiv 6 Jun 2021.

## Builds on (most cited first, 30 of 31)

- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [Sparsity Invariant CNNs](https://mlsanity.com/paper/1708.06500/) (2017)
- [Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraints](https://mlsanity.com/paper/1802.05522/) (2018)
- [Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Camera](https://mlsanity.com/paper/1807.00275/) (2018)
- [DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Image](https://mlsanity.com/paper/1812.00488/) (2018)
- [DeepPruner: Learning Efficient Stereo Matching via Differentiable PatchMatch](https://mlsanity.com/paper/1909.05845/) (2019)
- [Deep Depth Completion of a Single RGB-D Image](https://mlsanity.com/paper/1803.09326/) (2018)
- [CSPN++: Learning Context and Resource Aware Convolutional Spatial Propagation Networks for Depth Completion](https://mlsanity.com/paper/1911.05377/) (2019)
- [Depth Completion from Sparse LiDAR Data with Depth-Normal Constraints](https://mlsanity.com/paper/1910.06727/) (2019)
- [In Defense of Classical Image Processing: Fast Depth Completion on the CPU](https://mlsanity.com/paper/1802.00036/) (2018)
- [Unsupervised Depth Completion from Visual Inertial Odometry](https://mlsanity.com/paper/1905.08616/) (2019)
- [Sparse and noisy LiDAR completion with RGB guidance and uncertainty](https://mlsanity.com/paper/1902.05356/) (2019)
- [Sparse and Dense Data with CNNs: Depth Completion and Semantic Segmentation](https://mlsanity.com/paper/1808.00769/) (2018)
- [Learning Joint 2D-3D Representations for Depth Completion](https://mlsanity.com/paper/2012.12402/) (2020)
- [Dense Depth Posterior (DDP) from Single Image and Sparse Range](https://mlsanity.com/paper/1901.10034/) (2019)
- [Bilateral Cyclic Constraint and Adaptive Regularization for Unsupervised Monocular Depth Prediction](https://mlsanity.com/paper/1903.07309/) (2019)
- [Targeted Adversarial Perturbations for Monocular Depth Prediction](https://mlsanity.com/paper/2006.08602/) (2020)
- [DFuseNet: Deep Fusion of RGB and Sparse Depth Information for Image Guided Dense Depth Completion](https://mlsanity.com/paper/1902.00761/) (2019)
- [Propagating Confidences through CNNs for Sparse Data Regression](https://mlsanity.com/paper/1805.11913/) (2018)
- [Geo-Supervised Visual Depth Prediction](https://mlsanity.com/paper/1807.11130/) (2018)
- [HMS-Net: Hierarchical Multi-scale Sparsity-invariant Network for Sparse Depth Completion](https://mlsanity.com/paper/1808.08685/) (2018)
- [Deep Convolutional Compressed Sensing for LiDAR Depth Completion](https://mlsanity.com/paper/1803.08949/) (2018)
- [Every Pixel Counts: Unsupervised Geometry Learning with Holistic 3D Motion Understanding](https://mlsanity.com/paper/1806.10556/) (2018)
- [Depth Completion via Deep Basis Fitting](https://mlsanity.com/paper/1912.10336/) (2019)
- [CodeVIO: Visual-Inertial Odometry with Learned Optimizable Dense Depth](https://mlsanity.com/paper/2012.10133/) (2020)
- [Bayesian Deep Basis Fitting for Depth Completion with Uncertainty](https://mlsanity.com/paper/2103.15254/) (2021)
- [Improved Stereo Matching with Constant Highway Networks and Reflective Confidence Learning](https://mlsanity.com/paper/1701.00165/) (2016)
- [Deep Depth Estimation from Visual-Inertial SLAM](https://mlsanity.com/paper/2008.00092/) (2020)
- [Semantically Guided Depth Upsampling](https://mlsanity.com/paper/1608.00753/) (2016)
- [Adaptive Regularization in Convex Composite Optimization for Variational Imaging Problems](https://mlsanity.com/paper/1609.02356/) (2016)

## Built on by (most cited first, 30 of 30)

- [Unsupervised Depth Completion with Calibrated Backprojection Layers](https://mlsanity.com/paper/2108.10531/) (2021)
- [WorDepth: Variational Language Prior for Monocular Depth Estimation](https://mlsanity.com/paper/2404.03635/) (2024)
- [Deep Depth Completion from Extremely Sparse Data: A Survey](https://mlsanity.com/paper/2205.05335/) (2022)
- [Not Just Streaks: Towards Ground Truth for Single Image Deraining](https://mlsanity.com/paper/2206.10779/) (2022)
- [DesNet: Decomposed Scale-Consistent Network for Unsupervised Depth Completion](https://mlsanity.com/paper/2211.10994/) (2022)
- [Test-Time Adaptation for Depth Completion](https://mlsanity.com/paper/2402.03312/) (2024)
- [Monitored Distillation for Positive Congruent Depth Completion](https://mlsanity.com/paper/2203.16034/) (2022)
- [Multi-Modal Masked Pre-Training for Monocular Panoramic Depth Completion](https://mlsanity.com/paper/2203.09855/) (2022)
- [RSA: Resolving Scale Ambiguities in Monocular Depth Estimators through Language Descriptions](https://mlsanity.com/paper/2410.02924/) (2024)
- [Stereoscopic Universal Perturbations across Different Architectures and Datasets](https://mlsanity.com/paper/2112.06116/) (2021)
- [Robust Depth Completion with Uncertainty-Driven Loss Functions](https://mlsanity.com/paper/2112.07895/) (2021)
- [AugUndo: Scaling Up Augmentations for Monocular Depth Completion and Estimation](https://mlsanity.com/paper/2310.09739/) (2023)
- [Learnable Differencing Center for Nighttime Depth Perception](https://mlsanity.com/paper/2306.14538/) (2023)
- [Heteroscedastic Uncertainty Estimation Framework for Unsupervised Registration](https://mlsanity.com/paper/2312.00836/) (2023)
- [ProtoDepth: Unsupervised Continual Depth Completion with Prototypes](https://mlsanity.com/paper/2503.12745/) (2025)
- [Adaptive Correspondence Scoring for Unsupervised Medical Image Registration](https://mlsanity.com/paper/2312.00837/) (2023)
- [Extending Foundational Monocular Depth Estimators to Fisheye Cameras with Calibration Tokens](https://mlsanity.com/paper/2508.04928/) (2025)
- [All-day Depth Completion](https://mlsanity.com/paper/2405.17315/) (2024)
- [UnCLe: Benchmarking Unsupervised Continual Learning for Depth Completion](https://mlsanity.com/paper/2410.18074/) (2024)
- [Radar-Guided Polynomial Fitting for Metric Depth Estimation](https://mlsanity.com/paper/2503.17182/) (2025)
- [ETA: Energy-based Test-time Adaptation for Depth Completion](https://mlsanity.com/paper/2508.05989/) (2025)
- [Segmentation-guided Domain Adaptation for Efficient Depth Completion](https://mlsanity.com/paper/2210.09213/) (2022)
- [FCDSN-DC: An Accurate and Lightweight Convolutional Neural Network for Stereo Estimation with Depth Completion](https://mlsanity.com/paper/2209.06525/) (2022)
- [StarryGazer: Leveraging Monocular Depth Estimation Models for Domain-Agnostic Single Depth Image Completion](https://mlsanity.com/paper/2512.13147/) (2025)
- [VisTa3D: A Dataset and Benchmark for Thin Object Reconstruction from Vision, Tactile, and 3D Point Clouds](https://mlsanity.com/paper/2608.20740/) (2026)
- [On the Viability of Monocular Depth Pre-training for Semantic Segmentation](https://mlsanity.com/paper/2203.13987/) (2022)
- [MetaComp: Learning to Adapt for Online Depth Completion](https://mlsanity.com/paper/2207.10623/) (2022)
- [DEUX: Active Exploration for Learning Unsupervised Depth Perception](https://mlsanity.com/paper/2310.06164/) (2023)
- [Self-Supervised Depth Completion Guided by 3D Perception and Geometry Consistency](https://mlsanity.com/paper/2312.15263/) (2023)
- [From Perspective to Fisheye Depth Estimation and Open-Vocabulary Segmentation](https://mlsanity.com/paper/2608.27860/) (2026)

## Abstract (arXiv)

We present a method to infer a dense depth map from a color image and associated sparse depth measurements. Our main contribution lies in the design of an annealing process for determining co-visibility (occlusions, disocclusions) and the degree of regularization to impose on the model. We show that regularization and co-visibility are related via the fitness (residual) of model to data and both can be unified into a single framework to improve the learning process. Our method is an adaptive weighting scheme that guides optimization by measuring the residual at each pixel location over each training step for (i) estimating a soft visibility mask and (ii) determining the amount of regularization. We demonstrate the effectiveness our method by applying it to several recent unsupervised depth completion methods and improving their performance on public benchmark datasets, without incurring additional trainable parameters or increase in inference time. Code available at: https://github.com/alexklwong/adaframe-depth-completion.