mlsanityData to 14 Oct 2026

Pareto Multi-Task Learning

arXiv 1912.12854 ↗First posted 30 Dec 2019cs.LG stat.MLcited by 224

Not on the map. Why?

Abstract (arXiv)

Multi-task learning is a powerful method for solving multiple correlated tasks simultaneously. However, it is often impossible to find one single solution to optimize all the tasks, since different tasks might conflict with each other. Recently, a novel method is proposed to find one single Pareto optimal solution with good trade-off among different tasks by casting multi-task learning as multiobjective optimization. In this paper, we generalize this idea and propose a novel Pareto multi-task learning algorithm (Pareto MTL) to find a set of well-distributed Pareto solutions which can represent different trade-offs among different tasks. The proposed algorithm first formulates a multi-task learning problem as a multiobjective optimization problem, and then decomposes the multiobjective optimization problem into a set of constrained subproblems with different trade-off preferences. By solving these subproblems in parallel, Pareto MTL can find a set of well-representative Pareto optimal solutions with different trade-off among all tasks. Practitioners can easily select their preferred solution from these Pareto solutions, or use different trade-off solutions for different situations. Experimental results confirm that the proposed algorithm can generate well-representative solutions and outperform some state-of-the-art algorithms on many multi-task learning applications.

Key facts. Cited by 224 papers in the map (data through 14 Oct 2026). First posted on arXiv 30 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 34; 2022: 32; 2023: 38; 2024: 64; 2025: 37; 2026: 19.

224 papers.

Sort:

By area:

  1. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 459
  2. 2023Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspectivecited by 280
  3. 2021Efficiently Identifying Task Groupings for Multi-Task Learningcited by 222
  4. 2021Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Studycited by 168
  5. 2021Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learningcited by 101
  6. 2024C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersioncited by 83
  7. 2024Panacea: Pareto Alignment via Preference Adaptation for LLMscited by 80
  8. 2022Auto-Lambda: Disentangling Dynamic Task Relationshipscited by 73
  9. 2024MMPareto: Boosting Multimodal Learning with Innocent Unimodal Assistancecited by 73
  10. 2021Controlled Text Generation as Continuous Optimization with Multiple Constraintscited by 70
  11. 2022SummaReranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarizationcited by 61
  12. 2024Distribution Matching for Multi-Task Learning of Classification Tasks: a Large-Scale Study on Faces & Beyondcited by 60
38 more
  1. 2024Smooth Tchebycheff Scalarization for Multi-Objective Optimizationcited by 54
  2. 2021Pareto Self-Supervised Training for Few-Shot Learningcited by 52
  3. 2022Pareto Set Learning for Expensive Multi-Objective Optimizationcited by 51
  4. 2023Recon: Reducing Conflicting Gradients from the Root for Multi-Task Learningcited by 51
  5. 2023Revisiting Scalarization in Multi-Task Learning: A Theoretical Perspectivecited by 51
  6. 2021Addressing Algorithmic Disparity and Performance Inconsistency in Federated Learningcited by 50
  7. 2022PaCo: Parameter-Compositional Multi-Task Reinforcement Learningcited by 46
  8. 2021Scalable Pareto Front Approximation for Deep Multi-Objective Learningcited by 45
  9. 2021Same State, Different Task: Continual Reinforcement Learning without Interferencecited by 43
  10. 2021Learning Multiple Dense Prediction Tasks from Partially Annotated Datacited by 42
  11. 2021Fair Normalizing Flowscited by 38
  12. 2022One Model is All You Need: Multi-Task Learning Enables Simultaneous Histology Image Segmentation and Classificationcited by 38
  13. 2022Pareto Invariant Risk Minimization: Towards Mitigating the Optimization Dilemma in Out-of-Distribution Generalizationcited by 38
  14. 2023Three-Way Trade-Off in Multi-Objective Learning: Optimization, Generalization and Conflict-Avoidancecited by 36
  15. 2022Multi-task Self-supervised Graph Neural Networks Enable Stronger Task Generalizationcited by 35
  16. 2023Direction-oriented Multi-objective Learning: Simple and Provable Stochastic Algorithmscited by 34
  17. 2022AdaTask: A Task-aware Adaptive Learning Rate Approach to Multi-task Learningcited by 31
  18. 2022Pareto Manifold Learning: Tackling multiple tasks via ensembles of single-task modelscited by 29
  19. 2023Large Language Model for Multi-objective Evolutionary Optimizationcited by 29
  20. 2024Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generationcited by 27
  21. 2023Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mixture of Low-rank Expertscited by 26
  22. 2022Universal Representations: A Unified Look at Multiple Task and Domain Learningcited by 23
  23. 2024Delving into Multi-modal Multi-task Foundation Models for Road Scene Understanding: From Learning Paradigm Perspectivescited by 23
  24. 2022Debiasing Methods for Fairer Neural Models in Vision and Language Research: A Surveycited by 22
  25. 2022Controllable Dynamic Multi-Task Architecturescited by 21
  26. 2021Pareto Domain Adaptationcited by 20
  27. 2024When Large Language Models Meet Evolutionary Algorithms: Potential Enhancements and Challengescited by 20
  28. 2024MAP: Low-compute Model Merging with Amortized Pareto Fronts via Quadratic Approximationcited by 20
  29. 2025Pareto Set Learning for Multi-Objective Reinforcement Learningcited by 20
  30. 2023Scalarization for Multi-Task and Multi-Domain Learning at Scalecited by 19
  31. 2021Latent Space Smoothing for Individually Fair Representationscited by 18
  32. 2023Mitigating Task Interference in Multi-Task Learning via Explicit Task Routing with Non-Learnable Primitivescited by 18
  33. 2025Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyondcited by 18
  34. 2026Simulated Students in Tutoring Dialogues: Substance or Illusion?cited by 18
  35. 2021AutoMTL: A Programming Framework for Automating Efficient Multi-Task Learningcited by 17
  36. 2023Goal-conditioned GFlowNets for Controllable Multi-Objective Molecular Designcited by 17
  37. 2021Multi-Objective Learning to Predict Pareto Fronts Using Hypervolume Maximizationcited by 16
  38. 2021Sequential Reptile: Inter-Task Gradient Alignment for Multilingual Learningcited by 16

50 most cited of 224 All 224 (JSON)