# CompletionFormer: Depth Completion with Convolutions and Vision Transformers

arXiv: https://arxiv.org/abs/2304.13030 · HTML: https://mlsanity.com/paper/2304.13030/

## Key facts

- Cited by 93 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 53 references that are in the map.
- First posted on arXiv 25 Apr 2023.

## Builds on (most cited first, 30 of 45)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Sparsity Invariant CNNs](https://mlsanity.com/paper/1708.06500/) (2017)
- [CMT: Convolutional Neural Networks Meet Vision Transformers](https://mlsanity.com/paper/2107.06263/) (2021)
- [Learning to Estimate Hidden Motions with Global Motion Aggregation](https://mlsanity.com/paper/2104.02409/) (2021)
- [Co-Scale Conv-Attentional Image Transformers](https://mlsanity.com/paper/2104.06399/) (2021)
- [Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation](https://mlsanity.com/paper/2203.11483/) (2022)
- [Non-Local Spatial Propagation Network for Depth Completion](https://mlsanity.com/paper/2007.10042/) (2020)
- [Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2011.02910/) (2020)
- [Conformer: Local Features Coupling Global Representations for Visual Recognition](https://mlsanity.com/paper/2105.03889/) (2021)
- [Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image](https://mlsanity.com/paper/1709.07492/) (2017)
- [PENet: Towards Precise and Efficient Image Guided Depth Completion](https://mlsanity.com/paper/2103.00783/) (2021)
- [Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Camera](https://mlsanity.com/paper/1807.00275/) (2018)
- [MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformer](https://mlsanity.com/paper/2208.03543/) (2022)
- [DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Image](https://mlsanity.com/paper/1812.00488/) (2018)
- [MPViT: Multi-Path Vision Transformer for Dense Prediction](https://mlsanity.com/paper/2112.11010/) (2021)
- [DepthFormer: Exploiting Long-Range Correlation and Local Information for Accurate Monocular Depth Estimation](https://mlsanity.com/paper/2203.14211/) (2022)
- [Learning Guided Convolutional Network for Depth Completion](https://mlsanity.com/paper/1908.01238/) (2019)
- [Deep Depth Completion of a Single RGB-D Image](https://mlsanity.com/paper/1803.09326/) (2018)
- [Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network](https://mlsanity.com/paper/1808.00150/) (2018)
- [CSPN++: Learning Context and Resource Aware Convolutional Spatial Propagation Networks for Depth Completion](https://mlsanity.com/paper/1911.05377/) (2019)

## Built on by (most cited first, 30 of 93)

- [DepthFM: Fast Monocular Depth Estimation with Flow Matching](https://mlsanity.com/paper/2403.13788/) (2024)
- [Prompting Depth Anything for 4K Resolution Accurate Metric Depth Estimation](https://mlsanity.com/paper/2412.14015/) (2024)
- [Bilateral Propagation Network for Depth Completion](https://mlsanity.com/paper/2403.11270/) (2024)
- [Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusion](https://mlsanity.com/paper/2412.13389/) (2024)
- [LRRU: Long-short Range Recurrent Updating Networks for Depth Completion](https://mlsanity.com/paper/2310.08956/) (2023)
- [Tri-Perspective View Decomposition for Geometry-Aware Depth Completion](https://mlsanity.com/paper/2403.15008/) (2024)
- [SGD: Street View Synthesis with Gaussian Splatting and Diffusion Prior](https://mlsanity.com/paper/2403.20079/) (2024)
- [Depth Anything with Any Prior](https://mlsanity.com/paper/2505.10565/) (2025)
- [OMNI-DC: Highly Robust Depth Completion with Multiresolution Depth Integration](https://mlsanity.com/paper/2411.19278/) (2024)
- [Test-Time Adaptation for Depth Completion](https://mlsanity.com/paper/2402.03312/) (2024)
- [Depth Prompting for Sensor-Agnostic Depth Estimation](https://mlsanity.com/paper/2405.11867/) (2024)
- [DriveTrack: A Benchmark for Long-Range Point Tracking in Real-World Videos](https://mlsanity.com/paper/2312.09523/) (2023)
- [Invisible Stitch: Generating Smooth 3D Scenes with Depth Inpainting](https://mlsanity.com/paper/2404.19758/) (2024)
- [OGNI-DC: Robust Depth Completion with Optimization-Guided Neural Iterations](https://mlsanity.com/paper/2406.11711/) (2024)
- [Multi-view Reconstruction via SfM-guided Monocular Depth Estimation](https://mlsanity.com/paper/2503.14483/) (2025)
- [Revisiting Depth Completion from a Stereo Matching Perspective for Cross-domain Generalization](https://mlsanity.com/paper/2312.09254/) (2023)
- [CLAMP-ViT: Contrastive Data-Free Learning for Adaptive Post-Training Quantization of ViTs](https://mlsanity.com/paper/2407.05266/) (2024)
- [AugUndo: Scaling Up Augmentations for Monocular Depth Completion and Estimation](https://mlsanity.com/paper/2310.09739/) (2023)
- [Distilling Monocular Foundation Model for Fine-grained Depth Completion](https://mlsanity.com/paper/2503.16970/) (2025)
- [Learnable Differencing Center for Nighttime Depth Perception](https://mlsanity.com/paper/2306.14538/) (2023)
- [Sparse Beats Dense: Rethinking Supervision in Radar-Camera Depth Completion](https://mlsanity.com/paper/2312.00844/) (2023)
- [Completion as Enhancement: A Degradation-Aware Selective Image Guided Network for Depth Completion](https://mlsanity.com/paper/2412.19225/) (2024)
- [RigNet++: Semantic Assisted Repetitive Image Guided Network for Depth Completion](https://mlsanity.com/paper/2309.00655/) (2023)
- [Instruct Large Language Models to Drive like Humans](https://mlsanity.com/paper/2406.07296/) (2024)
- [Learning to Produce Semi-dense Correspondences for Visual Localization](https://mlsanity.com/paper/2402.08359/) (2024)
- [NDDepth: Normal-Distance Assisted Monocular Depth Estimation and Completion](https://mlsanity.com/paper/2311.07166/) (2023)
- [Active Stereo Without Pattern Projector](https://mlsanity.com/paper/2309.12315/) (2023)
- [SparseDC: Depth Completion from sparse and non-uniform inputs](https://mlsanity.com/paper/2312.00097/) (2023)
- [RGB Guided ToF Imaging System: A Survey of Deep Learning-based Methods](https://mlsanity.com/paper/2405.10357/) (2024)
- [All-day Depth Completion](https://mlsanity.com/paper/2405.17315/) (2024)

## Abstract (arXiv)

Given sparse depths and the corresponding RGB images, depth completion aims at spatially propagating the sparse measurements throughout the whole image to get a dense depth prediction. Despite the tremendous progress of deep-learning-based depth completion methods, the locality of the convolutional layer or graph model makes it hard for the network to model the long-range relationship between pixels. While recent fully Transformer-based architecture has reported encouraging results with the global receptive field, the performance and efficiency gaps to the well-developed CNN models still exist because of its deteriorative local feature details. This paper proposes a Joint Convolutional Attention and Transformer block (JCAT), which deeply couples the convolutional attention layer and Vision Transformer into one block, as the basic unit to construct our depth completion model in a pyramidal structure. This hybrid architecture naturally benefits both the local connectivity of convolutions and the global context of the Transformer in one single model. As a result, our CompletionFormer outperforms state-of-the-art CNNs-based methods on the outdoor KITTI Depth Completion benchmark and indoor NYUv2 dataset, achieving significantly higher efficiency (nearly 1/3 FLOPs) compared to pure Transformer-based methods. Code is available at urlhttps://github.com/youmi-zym/CompletionFormer.