mlsanityData to 7 Oct 2026

A Survey on Multi-Task Learning

arXiv 1707.08114 ↗First posted 25 Jul 2017cs.LG cs.AIcited by 838

Not on the map. Why?

Abstract (arXiv)

Multi-Task Learning (MTL) is a learning paradigm in machine learning and its aim is to leverage useful information contained in multiple related tasks to help improve the generalization performance of all the tasks. In this paper, we give a survey for MTL from the perspective of algorithmic modeling, applications and theoretical analyses. For algorithmic modeling, we give a definition of MTL and then classify different MTL algorithms into five categories, including feature learning approach, low-rank approach, task clustering approach, task relation learning approach and decomposition approach as well as discussing the characteristics of each approach. In order to improve the performance of learning tasks further, MTL can be combined with other learning paradigms including semi-supervised learning, active learning, unsupervised learning, reinforcement learning, multi-view learning and graphical models. When the number of tasks is large or the data dimensionality is high, we review online, parallel and distributed MTL models as well as dimensionality reduction and feature hashing to reveal their computational and storage advantages. Many real-world applications use MTL to boost their performance and we review representative works in this paper. Finally, we present theoretical analyses and discuss several future directions for MTL.

Key facts. Cited by 838 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Jul 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 133; 2022: 137; 2023: 164; 2024: 166; 2025: 155; 2026: 83.

838 papers.

Sort:

By area:

  1. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 795
  2. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 679
  3. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 441
  4. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 323
  5. 2021Efficiently Identifying Task Groupings for Multi-Task Learningcited by 218
  6. 2021Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysiscited by 179
  7. 2023FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Leadcited by 170
  8. 2021Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Studycited by 168
  9. 2023Large Language Models in Law: A Surveycited by 128
  10. 2023Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learningcited by 115
  11. 2023SAM-CLIP: Merging Vision Foundation Models towards Semantic and Spatial Understandingcited by 111
  12. 2023When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applicationscited by 110
38 more
  1. 2024EMR-Merging: Tuning-Free High-Performance Model Mergingcited by 108
  2. 2021Inducing Causal Structure for Interpretable Neural Networkscited by 107
  3. 2023Mixture of Cluster-conditional LoRA Experts for Vision-language Instruction Tuningcited by 106
  4. 2021Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learningcited by 100
  5. 2024UniTS: A Unified Multi-Task Time Series Modelcited by 93
  6. 2022Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?cited by 89
  7. 2022Physics-Informed Machine Learning: A Survey on Problems, Methods and Applicationscited by 88
  8. 2023Zenseact Open Dataset: A large-scale and diverse multimodal dataset for autonomous drivingcited by 83
  9. 2022Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challengescited by 81
  10. 2023A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4cited by 81
  11. 2023End-to-End Vectorized HD-map Construction with Piecewise Bezier Curvecited by 80
  12. 2023Federated Learning for Medical Image Analysis: A Surveycited by 77
  13. 2022Recent Advances in Bayesian Optimizationcited by 74
  14. 2021Replay in Deep Learning: Current Approaches and Missing Biological Elementscited by 71
  15. 20223D Object Detection from Images for Autonomous Driving: A Surveycited by 66
  16. 2023Advances and Challenges in Meta-Learning: A Technical Reviewcited by 65
  17. 2024Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learningcited by 63
  18. 2021Multi-Task Learning in Natural Language Processing: An Overviewcited by 61
  19. 2022Where in the World is this Image? Transformer-based Geo-localization in the Wildcited by 58
  20. 2021DEFT: Detection Embeddings for Trackingcited by 56
  21. 2021Deep Gait Recognition: A Surveycited by 54
  22. 2024FLaRe: Achieving Masterful and Adaptive Robot Policies with Large-Scale Reinforcement Learning Fine-Tuningcited by 54
  23. 2021Adapt to Adaptation: Learning Personalization for Cross-Silo Federated Learningcited by 53
  24. 2024MetaGPT: Merging Large Language Models Using Model Exclusive Task Arithmeticcited by 53
  25. 2022Multitask Vision-Language Prompt Tuningcited by 51
  26. 2023Revisiting Scalarization in Multi-Task Learning: A Theoretical Perspectivecited by 51
  27. 2024Localize-and-Stitch: Efficient Model Merging via Sparse Task Arithmeticcited by 50
  28. 2022CodeFill: Multi-token Code Completion by Jointly Learning from Structure and Naming Sequencescited by 49
  29. 2024Multi-Task Learning for Routing Problem with Cross-Problem Zero-Shot Generalizationcited by 49
  30. 2022MulT: An End-to-End Multitask Learning Transformercited by 47
  31. 2021Scalable Pareto Front Approximation for Deep Multi-Objective Learningcited by 45
  32. 2021A Survey of Sound Source Localization with Deep Learning Methodscited by 45
  33. 2023TaskExpert: Dynamically Assembling Multi-Task Representations with Memorial Mixture-of-Expertscited by 45
  34. 2023OmniVec: Learning robust representations with cross modal sharingcited by 44
  35. 2024MTLoRA: A Low-Rank Adaptation Approach for Efficient Multi-Task Learningcited by 44
  36. 2021Deep Learning for Micro-expression Recognition: A Surveycited by 43
  37. 2021Learning Multiple Dense Prediction Tasks from Partially Annotated Datacited by 42
  38. 2022PaCo: Parameter-Compositional Multi-Task Reinforcement Learningcited by 40

50 most cited of 838 All 838 (JSON)