mlsanityData to 14 Oct 2026

Unleashing the Power of Multi-Task Learning: A Comprehensive Survey Spanning Traditional, Deep, and Pretrained Foundation Model Eras

arXiv 2404.18961 ↗First posted 29 Apr 2024cs.LG cs.AI cs.CVcited by 26

Not on the map. Why?

Abstract (arXiv)

MTL is a learning paradigm that effectively leverages both task-specific and shared information to address multiple related tasks simultaneously. In contrast to STL, MTL offers a suite of benefits that enhance both the training process and the inference efficiency. MTL's key advantages encompass streamlined model architecture, performance enhancement, and cross-domain generalizability. Over the past twenty years, MTL has become widely recognized as a flexible and effective approach in various fields, including CV, NLP, recommendation systems, disease prognosis and diagnosis, and robotics. This survey provides a comprehensive overview of the evolution of MTL, encompassing the technical aspects of cutting-edge methods from traditional approaches to deep learning and the latest trend of pretrained foundation models. Our survey methodically categorizes MTL techniques into five key areas: regularization, relationship learning, feature propagation, optimization, and pre-training. This categorization not only chronologically outlines the development of MTL but also dives into various specialized strategies within each category. Furthermore, the survey reveals how the MTL evolves from handling a fixed set of tasks to embracing a more flexible approach free from task or modality constraints. It explores the concepts of task-promptable and -agnostic training, along with the capacity for ZSL, which unleashes the untapped potential of this historically coveted learning paradigm. Overall, we hope this survey provides the research community with a comprehensive overview of the advancements in MTL from its inception in 1997 to the present in 2023. We address present challenges and look ahead to future possibilities, shedding light on the opportunities and potential avenues for MTL research in a broad manner. This project is publicly available at https://github.com/junfish/Awesome-Multitask-Learning.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Apr 2024.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 14; 2026: 6.

26 papers.

Sort:

By area:

  1. 2025Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyondcited by 18
  2. 2024Swiss Army Knife: Synergizing Biases in Knowledge from Vision Foundation Models for Multi-Task Learningcited by 7
  3. 2025From Low Intrinsic Dimensionality to Non-Vacuous Generalization Bounds in Deep Multi-Task Learningcited by 5
  4. 2025FW-Merging: Scaling Model Merging with Frank-Wolfe Optimizationcited by 5
  5. 2024HM3: Heterogeneous Multi-Class Model Mergingcited by 3
  6. 2024Parameter-Efficient Interventions for Enhanced Model Mergingcited by 3
  7. 2025Mitigating Domain Drift in Multi Species Segmentation with DINOv2: A Cross-Domain Evaluation in Herbicide Research Trialscited by 3
  8. 2025One Model for All Tasks: Leveraging Efficient World Models in Multi-Task Planningcited by 3
  9. 2025Generalizable and Efficient Automated Scoring with a Knowledge-Distilled Multi-Task Mixture-of-Expertscited by 3
  10. 2024Multi-Type Preference Learning: Empowering Preference-Based Reinforcement Learning with Equal Preferencescited by 2
  11. 2024Exploring Model Kinship for Merging Large Language Modelscited by 2
  12. 20253D-Aware Multi-Task Learning with Cross-View Correlations for Dense Scene Understandingcited by 2
14 more
  1. 2024Stealthy Multi-Task Adversarial Attackscited by 1
  2. 2025Towards Unified Modeling in Federated Multi-Task Learning via Subspace Decouplingcited by 1
  3. 2025SAMO: A Lightweight Sharpness-Aware Approach for Multi-Task Optimization with Joint Global-Local Perturbationcited by 1
  4. 2025Surgical Video Understanding with Label Interpolationcited by 1
  5. 2025Parameter-Efficient Multi-Task Learning via Progressive Task-Specific Adaptationcited by 1
  6. 2025EarthSight: A Distributed Framework for Low-Latency Satellite Intelligencecited by 1
  7. 2026FLAME: Adaptive Mixture-of-Experts for Continual Multimodal Multi-Task Learningcited by 1
  8. 2025ColNet: Collaborative Optimization in Decentralized Federated Multi-task Learning Systemscited by 0
  9. 2025Reverse Probing: Evaluating Knowledge Transfer via Finetuned Task Embeddings for Coreference Resolutioncited by 0
  10. 2026Cascaded Transfer: Learning Many Tasks under Budget Constraintscited by 0
  11. 2026Taipan: A Query-free Transfer-based Multiple Sensitive Attribute Inference Attack Solely from Auxiliary Graphscited by 0
  12. 2026Meta-Learning and Meta-Reinforcement Learning -- Tracing the Path towards DeepMind's Adaptive Agentcited by 0
  13. 2026PRISM: Synergizing Vision Foundation Models via Self-organized Expert Specializationcited by 0
  14. 2026Gradient Surgery for Physics-Informed Neural Networkscited by 0