# Multi-task GANs for Semantic Segmentation and Depth Completion with Cycle Consistency

arXiv: https://arxiv.org/abs/2011.14272 · HTML: https://mlsanity.com/paper/2011.14272/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Nov 2020.

## Built on by (most cited first, 18 of 18)

- [Deep Depth Completion from Extremely Sparse Data: A Survey](https://mlsanity.com/paper/2205.05335/) (2022)
- [Unsupervised Monocular Depth Estimation in Highly Complex Environments](https://mlsanity.com/paper/2107.13137/) (2021)
- [SemAttNet: Towards Attention-based Semantic Aware Guided Depth Completion](https://mlsanity.com/paper/2204.13635/) (2022)
- [RGB-Depth Fusion GAN for Indoor Depth Completion](https://mlsanity.com/paper/2203.10856/) (2022)
- [Deep Attentional Guided Image Filtering](https://mlsanity.com/paper/2112.06401/) (2021)
- [Visual Semantic Segmentation Based on Few/Zero-Shot Learning: An Overview](https://mlsanity.com/paper/2211.08352/) (2022)
- [SwinMTL: A Shared Architecture for Simultaneous Depth Estimation and Semantic Segmentation from Monocular Camera Images](https://mlsanity.com/paper/2403.10662/) (2024)
- [RGB Guided ToF Imaging System: A Survey of Deep Learning-based Methods](https://mlsanity.com/paper/2405.10357/) (2024)
- [RDFC-GAN: RGB-Depth Fusion CycleGAN for Indoor Depth Completion](https://mlsanity.com/paper/2306.03584/) (2023)
- [Scale Propagation Network for Generalizable Depth Completion](https://mlsanity.com/paper/2410.18408/) (2024)
- [PacGDC: Label-Efficient Generalizable Depth Completion with Projection Ambiguity and Consistency](https://mlsanity.com/paper/2507.07374/) (2025)
- [Learn to Adapt for Monocular Depth Estimation](https://mlsanity.com/paper/2203.14005/) (2022)
- [A Threefold Review on Deep Semantic Segmentation: Efficiency-oriented, Temporal and Depth-aware design](https://mlsanity.com/paper/2303.04315/) (2023)
- [Progressive Depth Decoupling and Modulating for Flexible Depth Completion](https://mlsanity.com/paper/2405.09342/) (2024)
- [Single-Shot Metric Depth from Focused Plenoptic Cameras](https://mlsanity.com/paper/2412.02386/) (2024)
- [StarryGazer: Leveraging Monocular Depth Estimation Models for Domain-Agnostic Single Depth Image Completion](https://mlsanity.com/paper/2512.13147/) (2025)
- [Beyond Traditional Single Object Tracking: A Survey](https://mlsanity.com/paper/2405.10439/) (2024)
- [MT-Depth: Multi-task Instance feature analysis for the Depth Completion](https://mlsanity.com/paper/2512.04734/) (2025)

## Abstract (arXiv)

Semantic segmentation and depth completion are two challenging tasks in scene understanding, and they are widely used in robotics and autonomous driving. Although several works are proposed to jointly train these two tasks using some small modifications, like changing the last layer, the result of one task is not utilized to improve the performance of the other one despite that there are some similarities between these two tasks. In this paper, we propose multi-task generative adversarial networks (Multi-task GANs), which are not only competent in semantic segmentation and depth completion, but also improve the accuracy of depth completion through generated semantic images. In addition, we improve the details of generated semantic images based on CycleGAN by introducing multi-scale spatial pooling blocks and the structural similarity reconstruction loss. Furthermore, considering the inner consistency between semantic and geometric structures, we develop a semantic-guided smoothness loss to improve depth completion results. Extensive experiments on Cityscapes dataset and KITTI depth completion benchmark show that the Multi-task GANs are capable of achieving competitive performance for both semantic segmentation and depth completion tasks.