mlsanityData to 7 Oct 2026

An Overview of Multi-Task Learning in Deep Neural Networks

arXiv 1706.05098 ↗First posted 15 Jun 2017cs.LG cs.AI stat.MLcited by 791

Not on the map. Why?

Abstract (arXiv)

Multi-task learning (MTL) has led to successes in many applications of machine learning, from natural language processing and speech recognition to computer vision and drug discovery. This article aims to give a general overview of MTL, particularly in deep neural networks. It introduces the two most common methods for MTL in Deep Learning, gives an overview of the literature, and discusses recent advances. In particular, it seeks to help ML practitioners apply MTL by shedding light on how MTL works and providing guidelines for choosing appropriate auxiliary tasks.

Key facts. Cited by 791 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Jun 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 1; 2020: 1; 2021: 159; 2022: 150; 2023: 179; 2024: 115; 2025: 97; 2026: 88.

791 papers.

Sort:

By area:

  1. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  2. 2021MetaICL: Learning to Learn In Contextcited by 450
  3. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 441
  4. 2023Towards Generalist Biomedical AIcited by 286
  5. 2021Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworkscited by 278
  6. 2021Efficiently Identifying Task Groupings for Multi-Task Learningcited by 218
  7. 2022Multi-Task Learning as a Bargaining Gamecited by 209
  8. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusionscited by 208
  9. 2021Towards Causal Representation Learningcited by 207
  10. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  11. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  12. 2023Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretrainingcited by 162
38 more
  1. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 151
  2. 2024Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformerscited by 145
  3. 2021Improving Video-Text Retrieval by Multi-Stream Corpus Alignment and Dual Softmax Losscited by 116
  4. 2023Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learningcited by 115
  5. 2021DSelect-k: Differentiable Selection in the Mixture of Experts with Applications to Multi-Task Learningcited by 114
  6. 2021Inducing Causal Structure for Interpretable Neural Networkscited by 107
  7. 2023Synthesizing Physical Character-Scene Interactionscited by 102
  8. 2021SciFive: a text-to-text transformer model for biomedical literaturecited by 99
  9. 2022M3ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-designcited by 97
  10. 2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 90
  11. 2024Enhancing Noise Robustness of Retrieval-Augmented Language Models with Adaptive Adversarial Trainingcited by 88
  12. 2021Multi-Task Self-Training for Learning General Representationscited by 85
  13. 2022Tutorial on amortized optimizationcited by 83
  14. 2021RotoGrad: Gradient Homogenization in Multitask Learningcited by 82
  15. 2021Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmarkcited by 82
  16. 2023Modular Deep Learningcited by 82
  17. 2021One Model to Serve All: Star Topology Adaptive Recommender for Multi-Domain CTR Predictioncited by 78
  18. 2021ML-Decoder: Scalable and Versatile Classification Headcited by 78
  19. 2022Efficient Methods for Natural Language Processing: A Surveycited by 78
  20. 2022PEER: A Comprehensive and Multi-Task Benchmark for Protein Sequence Understandingcited by 77
  21. 2023Deep Learning for Survival Analysis: A Reviewcited by 75
  22. 2021On Transferability of Prompt Tuning for Natural Language Processingcited by 74
  23. 2021Replay in Deep Learning: Current Approaches and Missing Biological Elementscited by 71
  24. 2023Independent Component Alignment for Multi-Task Learningcited by 69
  25. 2022Exploring Dual-task Correlation for Pose Guided Person Image Generationcited by 68
  26. 2023vMAP: Vectorised Object Mapping for Neural Field SLAMcited by 67
  27. 2021AraT5: Text-to-Text Transformers for Arabic Language Generationcited by 63
  28. 2024Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learningcited by 63
  29. 2023Grounded Question-Answering in Long Egocentric Videoscited by 62
  30. 2022Where in the World is this Image? Transformer-based Geo-localization in the Wildcited by 58
  31. 2023Adapting Segment Anything Model for Change Detection in HR Remote Sensing Imagescited by 56
  32. 2023PIGEON: Predicting Image Geolocationscited by 55
  33. 2021Copula-based conformal prediction for Multi-Target Regressioncited by 53
  34. 20244M-21: An Any-to-Any Vision Model for Tens of Tasks and Modalitiescited by 52
  35. 2023Revisiting Scalarization in Multi-Task Learning: A Theoretical Perspectivecited by 51
  36. 2021Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptationcited by 50
  37. 2023Recon: Reducing Conflicting Gradients from the Root for Multi-Task Learningcited by 50
  38. 2022Generalizing to New Physical Systems via Context-Informed Dynamics Modelcited by 49

50 most cited of 791 All 791 (JSON)