mlsanityData to 7 Oct 2026

Multi-Task Learning with Deep Neural Networks: A Survey

arXiv 2009.09796 ↗First posted 10 Sep 2020cs.LG cs.CV stat.MLcited by 374

Not on the map. Why?

Abstract (arXiv)

Multi-task learning (MTL) is a subfield of machine learning in which multiple tasks are simultaneously learned by a shared model. Such approaches offer advantages like improved data efficiency, reduced overfitting through shared representations, and fast learning by leveraging auxiliary information. However, the simultaneous learning of multiple tasks presents new design and optimization challenges, and choosing which tasks should be learned jointly is in itself a non-trivial problem. In this survey, we give an overview of multi-task learning methods for deep neural networks, with the aim of summarizing both the well-established and most recent directions within the field. Our discussion is structured according to a partition of the existing deep MTL techniques into three groups: architectures, optimization methods, and task relationship learning. We also provide a summary of common multi-task benchmarks.

Key facts. Cited by 374 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Sep 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 41; 2022: 69; 2023: 83; 2024: 82; 2025: 63; 2026: 36.

374 papers.

Sort:

By area:

  1. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  2. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  3. 2022Planning-oriented Autonomous Drivingcited by 1,068
  4. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 679
  5. 2021ImageNet-21K Pretraining for the Massescited by 607
  6. 2023Multimodal Foundation Models: From Specialists to General-Purpose Assistantscited by 247
  7. 2023Efficient Diffusion Training via Min-SNR Weighting Strategycited by 242
  8. 2025CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillationcited by 216
  9. 2022Multi-Task Learning as a Bargaining Gamecited by 209
  10. 2021UniT: Multimodal Multitask Learning with a Unified Transformercited by 195
  11. 2023When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applicationscited by 110
  12. 2021Efficient Training of Visual Transformers with Small Datasetscited by 107
38 more
  1. 2021Inducing Causal Structure for Interpretable Neural Networkscited by 107
  2. 2023Modular Deep Learningcited by 82
  3. 2025A Survey on Vision-Language-Action Models for Autonomous Drivingcited by 72
  4. 2023Independent Component Alignment for Multi-Task Learningcited by 69
  5. 2023Advances and Challenges in Meta-Learning: A Technical Reviewcited by 65
  6. 2021Learning a Universal Template for Few-shot Dataset Generalizationcited by 61
  7. 2025EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editingcited by 59
  8. 2022Uni-Perceiver-MoE: Learning Sparse Generalist Models with Conditional MoEscited by 58
  9. 2021Deep Gait Recognition: A Surveycited by 54
  10. 2022Sparsely Activated Mixture-of-Experts are Robust Multi-Task Learnerscited by 51
  11. 2023Revisiting Scalarization in Multi-Task Learning: A Theoretical Perspectivecited by 51
  12. 2025UAVs Meet LLMs: Overviews and Perspectives Toward Agentic Low-Altitude Mobilitycited by 48
  13. 2025Large Language Models Hallucination: A Comprehensive Surveycited by 45
  14. 2022Watermarking Pre-trained Language Models with Backdooringcited by 44
  15. 2024MTLoRA: A Low-Rank Adaptation Approach for Efficient Multi-Task Learningcited by 44
  16. 2024The Perfect Blend: Redefining RLHF with Mixture of Judgescited by 43
  17. 2022A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methodscited by 42
  18. 2023Efficient RLHF: Reducing the Memory Usage of PPOcited by 40
  19. 2022One Model is All You Need: Multi-Task Learning Enables Simultaneous Histology Image Segmentation and Classificationcited by 38
  20. 2023FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion Taskscited by 37
  21. 2024Multimodal Instruction Tuning with Conditional Mixture of LoRAcited by 36
  22. 2021Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directionscited by 35
  23. 2022Intention-Conditioned Long-Term Human Egocentric Action Forecastingcited by 35
  24. 2023HealthFC: Verifying Health Claims with Evidence-Based Medical Fact-Checkingcited by 35
  25. 2023MFTCoder: Boosting Code LLMs with Multitask Fine-Tuningcited by 34
  26. 2021INTERN: A New Learning Paradigm Towards General Visioncited by 33
  27. 2021A Review on Deep Learning in UAV Remote Sensingcited by 32
  28. 2023FoodLMM: A Versatile Food Assistant using Large Multi-modal Modelcited by 32
  29. 2021Unified Detection of Digital and Physical Face Attackscited by 31
  30. 2024Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMscited by 31
  31. 2022Effective Adaptation in Multi-Task Co-Training for Unified Autonomous Drivingcited by 30
  32. 2022Pareto Manifold Learning: Tackling multiple tasks via ensembles of single-task modelscited by 29
  33. 2024Next Token Prediction Towards Multimodal Intelligence: A Comprehensive Surveycited by 27
  34. 2021Semantic Segmentation of Legal Documents via Rhetorical Rolescited by 26
  35. 2022Towards Versatile Embodied Navigationcited by 26
  36. 2023On Diversified Preferences of Large Language Model Alignmentcited by 26
  37. 2022Bidirectional Generation of Structure and Properties Through a Single Molecular Foundation Modelcited by 25
  38. 2024Neural Combinatorial Optimization Algorithms for Solving Vehicle Routing Problems: A Comprehensive Survey with Perspectivescited by 23

50 most cited of 374 All 374 (JSON)