# MT-Depth: Multi-task Instance feature analysis for the Depth Completion

arXiv: https://arxiv.org/abs/2512.04734 · HTML: https://mlsanity.com/paper/2512.04734/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 39 references that are in the map.
- First posted on arXiv 4 Dec 2025.

## Builds on (most cited first, 27 of 27)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [YOLOv11: An Overview of the Key Architectural Enhancements](https://mlsanity.com/paper/2410.17725/) (2024)
- [Virtual KITTI 2](https://mlsanity.com/paper/2001.10773/) (2020)
- [Virtual Worlds as Proxy for Multi-Object Tracking Analysis](https://mlsanity.com/paper/1605.06457/) (2016)
- [Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidance](https://mlsanity.com/paper/2007.06936/) (2020)
- [CSPN++: Learning Context and Resource Aware Convolutional Spatial Propagation Networks for Depth Completion](https://mlsanity.com/paper/1911.05377/) (2019)
- [OccDepth: A Depth-Aware Method for 3D Semantic Scene Completion](https://mlsanity.com/paper/2302.13540/) (2023)
- [SCPNet: Semantic Scene Completion on Point Cloud](https://mlsanity.com/paper/2303.06884/) (2023)
- [Learning Monocular Depth in Dynamic Scenes via Instance-Aware Projection Consistency](https://mlsanity.com/paper/2102.02629/) (2021)
- [Confidence Propagation through CNNs for Guided Sparse Depth Regression](https://mlsanity.com/paper/1811.01791/) (2018)
- [What is YOLOv8: An In-Depth Exploration of the Internal Features of the Next-Generation Object Detector](https://mlsanity.com/paper/2408.15857/) (2024)
- [DFuseNet: Deep Fusion of RGB and Sparse Depth Information for Image Guided Dense Depth Completion](https://mlsanity.com/paper/1902.00761/) (2019)
- [What is YOLOv5: A deep look into the internal features of the popular object detector](https://mlsanity.com/paper/2407.20892/) (2024)
- [Indoor Depth Completion with Boundary Consistency and Self-Attention](https://mlsanity.com/paper/1908.08344/) (2019)
- [Deep Learning for Multi-Task Medical Image Segmentation in Multiple Modalities](https://mlsanity.com/paper/1704.03379/) (2017)
- [Multi-task GANs for Semantic Segmentation and Depth Completion with Cycle Consistency](https://mlsanity.com/paper/2011.14272/) (2020)
- [Deep Learning-based Depth Estimation Methods from Monocular Image and Videos: A Comprehensive Survey](https://mlsanity.com/paper/2406.19675/) (2024)
- [Completion as Enhancement: A Degradation-Aware Selective Image Guided Network for Depth Completion](https://mlsanity.com/paper/2412.19225/) (2024)
- [Multi-task Learning with 3D-Aware Regularization](https://mlsanity.com/paper/2310.00986/) (2023)
- [Multi-task learning with cross-task consistency for improved depth estimation in colonoscopy](https://mlsanity.com/paper/2311.18664/) (2023)
- [CU-Net: LiDAR Depth-Only Completion With Coupled U-Net](https://mlsanity.com/paper/2210.14898/) (2022)
- [To complete or to estimate, that is the question: A Multi-Task Approach to Depth Completion and Monocular Depth Estimation](https://mlsanity.com/paper/1908.05540/) (2019)
- [SemSegDepth: A Combined Model for Semantic Segmentation and Depth Completion](https://mlsanity.com/paper/2209.00381/) (2022)
- [DGFusion: Depth-Guided Sensor Fusion for Robust Semantic Perception](https://mlsanity.com/paper/2509.09828/) (2025)
- [Deep Learning Techniques for Video Instance Segmentation: A Survey](https://mlsanity.com/paper/2310.12393/) (2023)
- [PanDepth: Joint Panoptic Segmentation and Depth Completion](https://mlsanity.com/paper/2212.14180/) (2022)

## Abstract (arXiv)

Depth completion plays a vital role in 3D perception systems, especially in scenarios where sparse depth data must be densified for tasks such as autonomous driving, robotics, and augmented reality. While many existing approaches rely on semantic segmentation to guide depth completion, they often overlook the benefits of object-level understanding. In this work, we introduce an instance-aware depth completion framework that explicitly integrates binary instance masks as spatial priors to refine depth predictions. Our model combines four main components: a frozen YOLO V11 instance segmentation branch, a U-Net-based depth completion backbone, a cross-attention fusion module, and an attention-guided prediction head. The instance segmentation branch generates per-image foreground masks that guide the depth branch via cross-attention, allowing the network to focus on object-centric regions during refinement. We validate our method on the Virtual KITTI 2 dataset, showing that it achieves lower Root Mean Squared Error (RMSE) compared to both a U-Net-only baseline and previous semantic-guided methods, while maintaining competitive Mean Absolute Error (MAE). Qualitative and quantitative results demonstrate that the proposed model effectively enhances depth accuracy near object boundaries, occlusions, and thin structures. Our findings suggest that incorporating instance-aware cues offers a promising direction for improving depth completion without relying on dense semantic labels.