# SemAttNet: Towards Attention-based Semantic Aware Guided Depth Completion

arXiv: https://arxiv.org/abs/2204.13635 · HTML: https://mlsanity.com/paper/2204.13635/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 51 references that are in the map.
- First posted on arXiv 28 Apr 2022.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [Sparsity Invariant CNNs](https://mlsanity.com/paper/1708.06500/) (2017)
- [Non-Local Spatial Propagation Network for Depth Completion](https://mlsanity.com/paper/2007.10042/) (2020)
- [Wider or Deeper: Revisiting the ResNet Model for Visual Recognition](https://mlsanity.com/paper/1611.10080/) (2016)
- [Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image](https://mlsanity.com/paper/1709.07492/) (2017)
- [PENet: Towards Precise and Efficient Image Guided Depth Completion](https://mlsanity.com/paper/2103.00783/) (2021)
- [Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Camera](https://mlsanity.com/paper/1807.00275/) (2018)
- [Learning Depth with Convolutional Spatial Propagation Network](https://mlsanity.com/paper/1810.02695/) (2018)
- [DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Image](https://mlsanity.com/paper/1812.00488/) (2018)
- [Augmented Reality Meets Computer Vision : Efficient Data Generation for Urban Driving Scenes](https://mlsanity.com/paper/1708.01566/) (2017)
- [Learning Guided Convolutional Network for Depth Completion](https://mlsanity.com/paper/1908.01238/) (2019)
- [CSPN++: Learning Context and Resource Aware Convolutional Spatial Propagation Networks for Depth Completion](https://mlsanity.com/paper/1911.05377/) (2019)
- [RigNet: Repetitive Image Guided Network for Depth Completion](https://mlsanity.com/paper/2107.13802/) (2021)
- [Learning Affinity via Spatial Propagation Networks](https://mlsanity.com/paper/1710.01020/) (2017)
- [Sparse and noisy LiDAR completion with RGB guidance and uncertainty](https://mlsanity.com/paper/1902.05356/) (2019)
- [Attention-Based Multimodal Fusion for Video Description](https://mlsanity.com/paper/1701.03126/) (2017)
- [Learning Joint 2D-3D Representations for Depth Completion](https://mlsanity.com/paper/2012.12402/) (2020)
- [Adaptive Context-Aware Multi-Modal Network for Depth Completion](https://mlsanity.com/paper/2008.10833/) (2020)
- [Uncertainty-Aware CNNs for Depth Completion: Uncertainty from Beginning to End](https://mlsanity.com/paper/2006.03349/) (2020)
- [FCFR-Net: Feature Fusion based Coarse-to-Fine Residual Learning for Depth Completion](https://mlsanity.com/paper/2012.08270/) (2020)
- [Depth Completion with Twin Surface Extrapolation at Occlusion Boundaries](https://mlsanity.com/paper/2104.02253/) (2021)
- [Deformable spatial propagation network for depth completion](https://mlsanity.com/paper/2007.04251/) (2020)
- [Deep Convolutional Compressed Sensing for LiDAR Depth Completion](https://mlsanity.com/paper/1803.08949/) (2018)
- [Depth Completion using Plane-Residual Representation](https://mlsanity.com/paper/2104.07350/) (2021)
- [Multi-task GANs for Semantic Segmentation and Depth Completion with Cycle Consistency](https://mlsanity.com/paper/2011.14272/) (2020)

## Built on by (most cited first, 23 of 23)

- [CompletionFormer: Depth Completion with Convolutions and Vision Transformers](https://mlsanity.com/paper/2304.13030/) (2023)
- [Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priors](https://mlsanity.com/paper/2503.17316/) (2025)
- [Enhancing Diffusion Models with 3D Perspective Geometry Constraints](https://mlsanity.com/paper/2312.00944/) (2023)
- [SACReg: Scene-Agnostic Coordinate Regression for Visual Localization](https://mlsanity.com/paper/2307.11702/) (2023)
- [FDCT: Fast Depth Completion for Transparent Objects](https://mlsanity.com/paper/2307.12274/) (2023)
- [RGB Guided ToF Imaging System: A Survey of Deep Learning-based Methods](https://mlsanity.com/paper/2405.10357/) (2024)
- [SteeredMarigold: Steering Diffusion Towards Depth Completion of Largely Incomplete Depth Maps](https://mlsanity.com/paper/2409.10202/) (2024)
- [Scale Propagation Network for Generalizable Depth Completion](https://mlsanity.com/paper/2410.18408/) (2024)
- [Masked Spatial Propagation Network for Sparsity-Adaptive Depth Refinement](https://mlsanity.com/paper/2404.19294/) (2024)
- [DeCoTR: Enhancing Depth Completion with 2D and 3D Attentions](https://mlsanity.com/paper/2403.12202/) (2024)
- [PacGDC: Label-Efficient Generalizable Depth Completion with Projection Ambiguity and Consistency](https://mlsanity.com/paper/2507.07374/) (2025)
- [DOC-Depth: A novel approach for dense depth ground truth generation](https://mlsanity.com/paper/2502.02144/) (2025)
- [Gated Cross-Attention Network for Depth Completion](https://mlsanity.com/paper/2309.16301/) (2023)
- [LiDAR Meta Depth Completion](https://mlsanity.com/paper/2307.12761/) (2023)
- [A Concise but High-performing Network for Image Guided Depth Completion in Autonomous Driving](https://mlsanity.com/paper/2401.15902/) (2024)
- [Progressive Depth Decoupling and Modulating for Flexible Depth Completion](https://mlsanity.com/paper/2405.09342/) (2024)
- [Single-Shot Metric Depth from Focused Plenoptic Cameras](https://mlsanity.com/paper/2412.02386/) (2024)
- [Perfecting Depth: Uncertainty-Aware Enhancement of Metric Depth](https://mlsanity.com/paper/2506.04612/) (2025)
- [StarryGazer: Leveraging Monocular Depth Estimation Models for Domain-Agnostic Single Depth Image Completion](https://mlsanity.com/paper/2512.13147/) (2025)
- [Temporal Lidar Depth Completion](https://mlsanity.com/paper/2406.11315/) (2024)
- [Revisiting Disparity from Dual-Pixel Images: Physics-Informed Lightweight Depth Estimation](https://mlsanity.com/paper/2411.04714/) (2024)
- [Training Self-Supervised Depth Completion Using Sparse Measurements and a Single Image](https://mlsanity.com/paper/2507.14845/) (2025)
- [Need for Speed: Zero-Shot Depth Completion with Single-Step Diffusion](https://mlsanity.com/paper/2603.10584/) (2026)

## Abstract (arXiv)

Depth completion involves recovering a dense depth map from a sparse map and an RGB image. Recent approaches focus on utilizing color images as guidance images to recover depth at invalid pixels. However, color images alone are not enough to provide the necessary semantic understanding of the scene. Consequently, the depth completion task suffers from sudden illumination changes in RGB images (e.g., shadows). In this paper, we propose a novel three-branch backbone comprising color-guided, semantic-guided, and depth-guided branches. Specifically, the color-guided branch takes a sparse depth map and RGB image as an input and generates color depth which includes color cues (e.g., object boundaries) of the scene. The predicted dense depth map of color-guided branch along-with semantic image and sparse depth map is passed as input to semantic-guided branch for estimating semantic depth. The depth-guided branch takes sparse, color, and semantic depths to generate the dense depth map. The color depth, semantic depth, and guided depth are adaptively fused to produce the output of our proposed three-branch backbone. In addition, we also propose to apply semantic-aware multi-modal attention-based fusion block (SAMMAFB) to fuse features between all three branches. We further use CSPN++ with Atrous convolutions to refine the dense depth map produced by our three-branch backbone. Extensive experiments show that our model achieves state-of-the-art performance in the KITTI depth completion benchmark at the time of submission.