# A Principled Approach for Learning Task Similarity in Multitask Learning

arXiv: https://arxiv.org/abs/1903.09109 · HTML: https://mlsanity.com/paper/1903.09109/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Mar 2019.

## Built on by (most cited first, 30 of 32)

- [Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learning](https://mlsanity.com/paper/2111.10603/) (2021)
- [A Survey on Deep Learning-Based Monocular Spacecraft Pose Estimation: Current State, Limitations and Prospects](https://mlsanity.com/paper/2305.07348/) (2023)
- [Aggregating From Multiple Target-Shifted Sources](https://mlsanity.com/paper/2105.04051/) (2021)
- [Scalarization for Multi-Task and Multi-Domain Learning at Scale](https://mlsanity.com/paper/2310.08910/) (2023)
- [Losses over Labels: Weakly Supervised Learning via Direct Loss Construction](https://mlsanity.com/paper/2212.06921/) (2022)
- [Similarity of Classification Tasks](https://mlsanity.com/paper/2101.11201/) (2021)
- [A Taxonomy of Similarity Metrics for Markov Decision Processes](https://mlsanity.com/paper/2103.04706/) (2021)
- [Multitask Online Mirror Descent](https://mlsanity.com/paper/2106.02393/) (2021)
- [Multi-task Learning by Leveraging the Semantic Information](https://mlsanity.com/paper/2103.02546/) (2021)
- [Probabilistic task modelling for meta-learning](https://mlsanity.com/paper/2106.04802/) (2021)
- [Gap Minimization for Knowledge Sharing and Transfer](https://mlsanity.com/paper/2201.11231/) (2022)
- [Exploring Dark Knowledge under Various Teacher Capacities and Addressing Capacity Mismatch](https://mlsanity.com/paper/2405.13078/) (2024)
- [PhysioMTL: Personalizing Physiological Patterns using Optimal Transport Multi-Task Regression](https://mlsanity.com/paper/2203.12595/) (2022)
- [Provable Pathways: Learning Multiple Tasks over Multiple Paths](https://mlsanity.com/paper/2303.04338/) (2023)
- [Redefining non-IID Data in Federated Learning for Computer Vision Tasks: Migrating from Labels to Embeddings for Task-Specific Data Distributions](https://mlsanity.com/paper/2503.14553/) (2025)
- [Efficient Extraction of Pathologies from C-Spine Radiology Reports using Multi-Task Learning](https://mlsanity.com/paper/2204.04544/) (2022)
- [Task Weighting in Meta-learning with Trajectory Optimisation](https://mlsanity.com/paper/2301.01400/) (2023)
- [What is the Right Notion of Distance between Predict-then-Optimize Tasks?](https://mlsanity.com/paper/2409.06997/) (2024)
- [MINT: Multimodal Instruction Tuning with Multimodal Interaction Grouping](https://mlsanity.com/paper/2506.02308/) (2025)
- [Explaining the Effectiveness of Multi-Task Learning for Efficient Knowledge Extraction from Spine MRI Reports](https://mlsanity.com/paper/2205.02979/) (2022)
- [AdaTask: Adaptive Multitask Online Learning](https://mlsanity.com/paper/2205.15802/) (2022)
- [Can Similarity-Based Domain-Ordering Reduce Catastrophic Forgetting for Intent Recognition?](https://mlsanity.com/paper/2402.14155/) (2024)
- [Statistical Deficiency for Task Inclusion Estimation](https://mlsanity.com/paper/2503.05491/) (2025)
- [Recovery Guarantees for Continual Learning of Dependent Tasks: Memory, Data-Dependent Regularization, and Data-Dependent Weights](https://mlsanity.com/paper/2604.17578/) (2026)
- [Regressing Relative Fine-Grained Change for Sub-Groups in Unreliable Heterogeneous Data Through Deep Multi-Task Metric Learning](https://mlsanity.com/paper/2208.05800/) (2022)
- [Model-based adaptation for sample efficient transfer in reinforcement learning control of parameter-varying systems](https://mlsanity.com/paper/2305.12158/) (2023)
- [Enabling Asymmetric Knowledge Transfer in Multi-Task Learning with Self-Auxiliaries](https://mlsanity.com/paper/2410.15875/) (2024)
- [TLXML: Task-Level Explanation of Meta-Learning via Influence Functions](https://mlsanity.com/paper/2501.14271/) (2025)
- [Personalized Federated Learning under Model Dissimilarity Constraints](https://mlsanity.com/paper/2505.07575/) (2025)
- [Task-Aware Multi-Expert Architecture For Lifelong Deep Learning](https://mlsanity.com/paper/2512.11243/) (2025)

## Abstract (arXiv)

Multitask learning aims at solving a set of related tasks simultaneously, by exploiting the shared knowledge for improving the performance on individual tasks. Hence, an important aspect of multitask learning is to understand the similarities within a set of tasks. Previous works have incorporated this similarity information explicitly (e.g., weighted loss for each task) or implicitly (e.g., adversarial loss for feature adaptation), for achieving good empirical performances. However, the theoretical motivations for adding task similarity knowledge are often missing or incomplete. In this paper, we give a different perspective from a theoretical point of view to understand this practice. We first provide an upper bound on the generalization error of multitask learning, showing the benefit of explicit and implicit task similarity knowledge. We systematically derive the bounds based on two distinct task similarity metrics: H divergence and Wasserstein distance. From these theoretical results, we revisit the Adversarial Multi-task Neural Network, proposing a new training algorithm to learn the task relation coefficients and neural network parameters iteratively. We assess our new algorithm empirically on several benchmarks, showing not only that we find interesting and robust task relations, but that the proposed approach outperforms the baselines, reaffirming the benefits of theoretical insight in algorithm design.