mlsanityData to 14 Oct 2026

On Steering Multi-Annotations per Sample for Multi-Task Learning

arXiv 2203.02946 ↗First posted 6 Mar 2022cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The study of multi-task learning has drawn great attention from the community. Despite the remarkable progress, the challenge of optimally learning different tasks simultaneously remains to be explored. Previous works attempt to modify the gradients from different tasks. Yet these methods give a subjective assumption of the relationship between tasks, and the modified gradient may be less accurate. In this paper, we introduce Stochastic Task Allocation~(STA), a mechanism that addresses this issue by a task allocation approach, in which each sample is randomly allocated a subset of tasks. For further progress, we propose Interleaved Stochastic Task Allocation~(ISTA) to iteratively allocate all tasks to each example during several consecutive iterations. We evaluate STA and ISTA on various datasets and applications: NYUv2, Cityscapes, and COCO for scene understanding and instance segmentation. Our experiments show both STA and ISTA outperform current state-of-the-art methods. The code will be available.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 44 references that are in the map. First posted on arXiv 6 Mar 2022.

Builds on

33 of 44 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2017Mask R-CNNcited by 7,277
  4. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  5. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  6. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  7. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,487
  8. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
  9. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  10. 2018Efficient Lifelong Learning with A-GEMcited by 1,058
  11. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 834
  12. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
21 more
  1. 2018Taskonomy: Disentangling Task Transfer Learningcited by 634
  2. 2018End-to-End Multi-Task Learning with Attentioncited by 534
  3. 2016Cross-stitch Networks for Multi-task Learningcited by 467
  4. 2019Multi-Task Deep Neural Networks for Natural Language Understandingcited by 350
  5. 2018PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsingcited by 224
  6. 2020Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropoutcited by 201
  7. 2020MTI-Net: Multi-Scale Task Interaction Networks for Multi-Task Learningcited by 157
  8. 2019Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentationcited by 142
  9. 2018Multi-task Deep Reinforcement Learning with PopArtcited by 141
  10. 2018Adapting Auxiliary Losses Using Gradient Similaritycited by 100
  11. 2021VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluationcited by 95
  12. 2021YOLOP: You Only Look Once for Panoptic Driving Perceptioncited by 82
  13. 2018A Modulation Module for Multi-task Learning with Applications in Image Retrievalcited by 49
  14. 2019Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understandingcited by 37
  15. 2019Regularizing Deep Multi-Task Networks using Orthogonal Gradientscited by 36
  16. 2021Towards robust vision by multi-task learning on monkey visual cortexcited by 28
  17. 2021Deep Multi-Task Learning for Joint Localization, Perception, and Predictioncited by 22
  18. 2016When is multitask learning effective? Semantic sequence prediction under varying data conditionscited by 21
  19. 2019Learning Task Relatedness in Multi-Task Learning for Images in Contextcited by 9
  20. 2021SLAW: Scaled Loss Approximate Weighting for Efficient Multi-Task Learningcited by 3
  21. 2021Multi-task learning on the edge: cost-efficiency and theoretical optimalitycited by 1

Built on by

Not cited yet.