mlsanityData to 14 Oct 2026

Semi-supervised Multi-task Learning for Semantics and Depth

arXiv 2110.07197 ↗First posted 14 Oct 2021cs.CVcited by 22

Not on the map. Why?

Abstract (arXiv)

Multi-Task Learning (MTL) aims to enhance the model generalization by sharing representations between related tasks for better performance. Typical MTL methods are jointly trained with the complete multitude of ground-truths for all tasks simultaneously. However, one single dataset may not contain the annotations for each task of interest. To address this issue, we propose the Semi-supervised Multi-Task Learning (SemiMTL) method to leverage the available supervisory signals from different datasets, particularly for semantic segmentation and depth estimation tasks. To this end, we design an adversarial learning scheme in our semi-supervised training by leveraging unlabeled data to optimize all the task branches simultaneously and accomplish all tasks across datasets with partial annotations. We further present a domain-aware discriminator structure with various alignment formulations to mitigate the domain discrepancy issue among datasets. Finally, we demonstrate the effectiveness of the proposed method to learn across different datasets on challenging street view and remote sensing benchmarks.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 49 of its 73 references that are in the map. First posted on arXiv 14 Oct 2021.

Builds on

49 of 73 references.

References · 49

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  4. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  5. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  6. 2016Learning without Forgettingcited by 2,496
  7. 2016Pyramid Scene Parsing Networkcited by 2,479
  8. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
  9. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
  10. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
  11. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  12. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
37 more
  1. 2017A Survey on Multi-Task Learningcited by 840
  2. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 834
  3. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
  4. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 820
  5. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 803
  6. 2018Taskonomy: Disentangling Task Transfer Learningcited by 634
  7. 2018Learning to Adapt Structured Output Space for Semantic Segmentationcited by 584
  8. 2016Cross-stitch Networks for Multi-task Learningcited by 467
  9. 2016Deeper Depth Prediction with Fully Convolutional Residual Networkscited by 403
  10. 2018Context Encoding for Semantic Segmentationcited by 290
  11. 2015Semi-Supervised Learning with Ladder Networkscited by 247
  12. 2018PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsingcited by 224
  13. 2016UberNet: Training a `Universal' Convolutional Neural Network for Low-, Mid-, and High-Level Vision using Diverse Datasets and Limited Memorycited by 200
  14. 2018Adversarial Learning for Semi-Supervised Semantic Segmentationcited by 173
  15. 2019FickleNet: Weakly and Semi-supervised Semantic Image Segmentation using Stochastic Inferencecited by 162
  16. 2020MTI-Net: Multi-Scale Task Interaction Networks for Multi-Task Learningcited by 157
  17. 2018DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistencycited by 148
  18. 2019Attentive Single-Tasking of Multiple Taskscited by 148
  19. 2019Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentationcited by 142
  20. 2018Synscapes: A Photorealistic Synthetic Dataset for Street Scene Parsingcited by 106
  21. 2018Structured Attention Guided Convolutional Neural Fields for Monocular Depth Estimationcited by 78
  22. 2021InverseForm: A Loss Function for Structured Boundary-Aware Segmentationcited by 65
  23. 2017Cross-Domain Self-supervised Multi-task Feature Learning using Synthetic Imagerycited by 56
  24. 2018AdaDepth: Unsupervised Content Congruent Adaptation for Depth Estimationcited by 54
  25. 2020Joint Semantic Segmentation and Boundary Detection using Iterative Pyramid Contextscited by 51
  26. 2017Smooth Neighbors on Teacher Graphs for Semi-supervised Learningcited by 48
  27. 2018Geometry meets semantics for semi-supervised monocular depth estimationcited by 29
  28. 2021When Age-Invariant Face Recognition Meets Face Age Synthesis: A Multi-Task Learning Frameworkcited by 29
  29. 2015Decoupled Deep Neural Network for Semi-supervised Semantic Segmentationcited by 23
  30. 2019Bridging Stereo Matching and Optical Flow via Spatiotemporal Correspondencecited by 23
  31. 2020Densely connected multidilated convolutional networks for dense prediction taskscited by 21
  32. 2016Multi-Task Learning with Labeled and Unlabeled Taskscited by 17
  33. 2019Learning Across Tasks and Domainscited by 16
  34. 2017Scene Parsing with Global Context Embeddingcited by 7
  35. 2019Veritatem Dies Aperit- Temporally Consistent Depth Prediction Enabled by a Multi-Task Geometric and Semantic Scene Understanding Approachcited by 4
  36. 2021Instance-Level Task Parameters: A Robust Multi-task Weighting Frameworkcited by 4
  37. 2017Adversarial Learning of Structure-Aware Fully Convolutional Networks for Landmark Localizationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 7; 2024: 5; 2025: 6; 2026: 2.

22 papers.

Sort:

By area:

  1. 2023Loss Functions in the Era of Semantic Segmentation: A Survey and Outlookcited by 34
  2. 2022DenseMTL: Cross-task Attention Mechanism for Dense Multi-task Learningcited by 25
  3. 2024PhysMLE: Generalizable and Priors-Inclusive Multi-task Remote Physiological Measurementcited by 10
  4. 2022Mind The Edge: Refining Depth Edges in Sparsely-Supervised Monocular Depth Estimationcited by 8
  5. 2023Depth Estimation and Image Restoration by Deep Learning from Defocused Imagescited by 7
  6. 2023When Multi-Task Learning Meets Partial Supervision: A Computer Vision Reviewcited by 6
  7. 2024Joint-Task Regularization for Partially Labeled Multi-Task Learningcited by 6
  8. 2025Synchronizing Task Behavior: Aligning Multiple Tasks during Test-Time Trainingcited by 5
  9. 2024Multi-Task Label Discovery via Hierarchical Task Tokens for Partially Annotated Dense Predictionscited by 4
  10. 2025Align the GAP: Prior-based Unified Multi-Task Remote Physiological Measurement Framework For Domain Generalization and Personalizationcited by 4
  11. 2024Multi-task Geometric Estimation of Depth and Surface Normal from Monocular 360deg Imagescited by 3
  12. 2025Revisit the Imbalance Optimization in Multi-task Learning: An Experimental Analysiscited by 3
10 more
  1. 2023A Dynamic Feature Interaction Framework for Multi-task Visual Perceptioncited by 2
  2. 2023SOccDPT: Semi-Supervised 3D Semantic Occupancy from Dense Prediction Transformers trained under memory constraintscited by 2
  3. 2024Boosting Box-supervised Instance Segmentation with Pseudo Depthcited by 2
  4. 2025StableMTL: Repurposing Latent Diffusion Models for Multi-Task Learning from Partially Annotated Synthetic Datasetscited by 2
  5. 2023Knowledge Assembly: Semi-Supervised Multi-Task Learning from Multiple Datasets with Disjoint Labelscited by 1
  6. 2023Achieving RGB-D level Segmentation Performance from a Single ToF Cameracited by 0
  7. 2025Semi-Supervised Multi-Task Learning for Interpretable Quality As- sessment of Fundus Imagescited by 0
  8. 2025NexusFlow: Unifying Disparate Tasks under Partial Supervision via Invertible Flow Networkscited by 0
  9. 2026Multi-Task Learning for Robot Perception with Imbalanced Datacited by 0
  10. 2026Multi-task learning on partially labeled datasets via invariant/equivariant semi-supervised learningcited by 0