mlsanityData to 7 Oct 2026

Which Tasks Should Be Learned Together in Multi-task Learning?

arXiv 1905.07553 ↗First posted 18 May 2019cs.CVcited by 367

Not on the map. Why?

Abstract (arXiv)

Many computer vision applications require solving multiple tasks in real-time. A neural network can be trained to solve multiple tasks simultaneously using multi-task learning. This can save computation at inference time as only a single network needs to be evaluated. Unfortunately, this often leads to inferior overall performance as task objectives can compete, which consequently poses the question: which tasks should and should not be learned together in one network when employing multi-task learning? We study task cooperation and competition in several different learning settings and propose a framework for assigning tasks to a few neural networks such that cooperating tasks are computed by the same neural network, while competing tasks are computed by different networks. Our framework offers a time-accuracy trade-off and can produce better accuracy using less inference time than not only a single large multi-task neural network but also many single-task networks.

Key facts. Cited by 367 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 49; 2022: 56; 2023: 65; 2024: 63; 2025: 69; 2026: 65.

367 papers.

Sort:

By area:

  1. 2023AdaMerging: Adaptive Model Merging for Multi-Task Learningcited by 248
  2. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  3. 2021Efficiently Identifying Task Groupings for Multi-Task Learningcited by 218
  4. 2022Multi-Task Learning as a Bargaining Gamecited by 209
  5. 2021UniT: Multimodal Multitask Learning with a Unified Transformercited by 195
  6. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  7. 2021CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLPcited by 142
  8. 2024Localizing Task Information for Improved Model Merging and Compressioncited by 130
  9. 2022M3ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-designcited by 97
  10. 2022Instruction-driven history-aware policies for robotic manipulationscited by 93
  11. 2023Deep Model Fusion: A Surveycited by 89
  12. 2021RotoGrad: Gradient Homogenization in Multitask Learningcited by 82
38 more
  1. 2023FAMO: Fast Adaptive Multitask Optimizationcited by 74
  2. 2022Auto-Lambda: Disentangling Dynamic Task Relationshipscited by 73
  3. 2021Replay in Deep Learning: Current Approaches and Missing Biological Elementscited by 71
  4. 2026MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Trainingcited by 60
  5. 2022Uni-Perceiver-MoE: Learning Sparse Generalist Models with Conditional MoEscited by 58
  6. 2021Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Taskcited by 54
  7. 20244M-21: An Any-to-Any Vision Model for Tens of Tasks and Modalitiescited by 52
  8. 2024A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learningcited by 52
  9. 2022Multitask Vision-Language Prompt Tuningcited by 51
  10. 2022Meta-DMoE: Adapting to Domain Shift by Meta-Distillation from Mixture-of-Expertscited by 50
  11. 2023Recon: Reducing Conflicting Gradients from the Root for Multi-Task Learningcited by 50
  12. 2022A Closer Look at Few-shot Image Generationcited by 49
  13. 2022MulT: An End-to-End Multitask Learning Transformercited by 47
  14. 2021Scalable Pareto Front Approximation for Deep Multi-Objective Learningcited by 45
  15. 2023OmniVec: Learning robust representations with cross modal sharingcited by 44
  16. 2022A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methodscited by 42
  17. 2024PoCo: Policy Composition from and for Heterogeneous Robot Learningcited by 42
  18. 2023MmAP : Multi-modal Alignment Prompt for Cross-domain Multi-task Learningcited by 41
  19. 2025XiYan-SQL: A Novel Multi-Generator Framework For Text-to-SQLcited by 40
  20. 2022The Neural Race Reduction: Dynamics of Abstraction in Gated Networkscited by 37
  21. 2022Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learningcited by 36
  22. 2021Understanding and Improving Fairness-Accuracy Trade-offs in Multi-Task Learningcited by 34
  23. 2023Addressing Negative Transfer in Diffusion Modelscited by 33
  24. 2021Learning to Relate Depth and Semantics for Unsupervised Domain Adaptationcited by 32
  25. 2022Efficient Multi-Task RGB-D Scene Analysis for Indoor Environmentscited by 32
  26. 2023Sibling-Attack: Rethinking Transferable Adversarial Attacks against Face Recognitioncited by 32
  27. 2023Leveraging sparse and shared feature activations for disentangled representation learningcited by 32
  28. 2022High-Modality Multimodal Transformer: Quantifying Modality & Interaction Heterogeneity for High-Modality Representation Learningcited by 31
  29. 2023Test-Time Domain Adaptation by Learning Domain-Aware Batch Normalizationcited by 31
  30. 2022Effective Adaptation in Multi-Task Co-Training for Unified Autonomous Drivingcited by 30
  31. 2022Pareto Manifold Learning: Tackling multiple tasks via ensembles of single-task modelscited by 29
  32. 2023Universal Few-shot Learning of Dense Prediction Tasks with Visual Token Matchingcited by 29
  33. 2021CrossLoc: Scalable Aerial Localization Assisted by Multimodal Synthetic Datacited by 28
  34. 2022Improving Multi-Task Generalization via Regularizing Spurious Correlationcited by 28
  35. 2023ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learningcited by 28
  36. 2023Prismer: A Vision-Language Model with Multi-Task Expertscited by 28
  37. 2023Dual-Balancing for Multi-Task Learningcited by 27
  38. 2022DenseMTL: Cross-task Attention Mechanism for Dense Multi-task Learningcited by 25

50 most cited of 367 All 367 (JSON)