SparseFormer: Attention-based Depth Completion Network
Not on the map. Why?
Abstract (arXiv)
Most pipelines for Augmented and Virtual Reality estimate the ego-motion of the camera by creating a map of sparse 3D landmarks. In this paper, we tackle the problem of depth completion, that is, densifying this sparse 3D map using RGB images as guidance. This remains a challenging problem due to the low density, non-uniform and outlier-prone 3D landmarks produced by SfM and SLAM pipelines. We introduce a transformer block, SparseFormer, that fuses 3D landmarks with deep visual features to produce dense depth. The SparseFormer has a global receptive field, making the module especially effective for depth completion with low-density and non-uniform landmarks. To address the issue of depth outliers among the 3D landmarks, we introduce a trainable refinement module that filters outliers through attention between the sparse landmarks.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 21 of its 25 references that are in the map. First posted on arXiv 9 Jun 2022.
Builds on
21 of 25 references.
References · 21
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2020End-to-End Object Detection with Transformerscited by 7,384
- 2021Vision Transformers for Dense Predictioncited by 1,789
- 2020Reformer: The Efficient Transformercited by 1,658
- 2020Rethinking Attention with Performerscited by 1,584
- 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
- 2017Sparsity Invariant CNNscited by 325
- 2020Non-Local Spatial Propagation Network for Depth Completioncited by 194
- 2019Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Drivingcited by 188
- 2017Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Imagecited by 159
- 2021PENet: Towards Precise and Efficient Image Guided Depth Completioncited by 146
- 2018Learning Depth with Convolutional Spatial Propagation Networkcited by 135
9 more
- 2019Learning Guided Convolutional Network for Depth Completioncited by 111
- 2019CSPN++: Learning Context and Resource Aware Convolutional Spatial Propagation Networks for Depth Completioncited by 110
- 2021RigNet: Repetitive Image Guided Network for Depth Completioncited by 80
- 2017Learning Affinity via Spatial Propagation Networkscited by 79
- 2020Uncertainty-Aware CNNs for Depth Completion: Uncertainty from Beginning to Endcited by 55
- 2018Confidence Propagation through CNNs for Guided Sparse Depth Regressioncited by 52
- 2018Propagating Confidences through CNNs for Sparse Data Regressioncited by 36
- 2018Estimating Depth from RGB and Sparse Sensingcited by 23
- 2021Self-Supervised Depth Completion for Active Stereocited by 3
Built on by
Citing papers per year: 2023: 2.
2 papers.