# Maximum Roaming Multi-Task Learning

arXiv: https://arxiv.org/abs/2006.09762 · HTML: https://mlsanity.com/paper/2006.09762/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jun 2020.

## Built on by (most cited first, 11 of 11)

- [Denoising Task Routing for Diffusion Models](https://mlsanity.com/paper/2310.07138/) (2023)
- [Mitigating Task Interference in Multi-Task Learning via Explicit Task Routing with Non-Learnable Primitives](https://mlsanity.com/paper/2308.02066/) (2023)
- [Security Vulnerability Detection with Multitask Self-Instructed Fine-Tuning of Large Language Models](https://mlsanity.com/paper/2406.05892/) (2024)
- [Improved optimization strategies for deep Multi-Task Networks](https://mlsanity.com/paper/2109.11678/) (2021)
- [Multi-Task Structural Learning using Local Task Similarity induced Neuron Creation and Removal](https://mlsanity.com/paper/2305.00441/) (2023)
- [Deep Feature Surgery: Towards Accurate and Efficient Multi-Exit Networks](https://mlsanity.com/paper/2407.13986/) (2024)
- [Deep Safe Multi-Task Learning](https://mlsanity.com/paper/2111.10601/) (2021)
- ["It's a Match!" -- A Benchmark of Task Affinity Scores for Joint Learning](https://mlsanity.com/paper/2301.02873/) (2023)
- [Challenges and Opportunities of Using Transformer-Based Multi-Task Learning in NLP Through ML Lifecycle: A Survey](https://mlsanity.com/paper/2308.08234/) (2023)
- [Task adaption by biologically inspired stochastic comodulation](https://mlsanity.com/paper/2311.15053/) (2023)
- [Enabling Asymmetric Knowledge Transfer in Multi-Task Learning with Self-Auxiliaries](https://mlsanity.com/paper/2410.15875/) (2024)

## Abstract (arXiv)

Multi-task learning has gained popularity due to the advantages it provides with respect to resource usage and performance. Nonetheless, the joint optimization of parameters with respect to multiple tasks remains an active research topic. Sub-partitioning the parameters between different tasks has proven to be an efficient way to relax the optimization constraints over the shared weights, may the partitions be disjoint or overlapping. However, one drawback of this approach is that it can weaken the inductive bias generally set up by the joint task optimization. In this work, we present a novel way to partition the parameter space without weakening the inductive bias. Specifically, we propose Maximum Roaming, a method inspired by dropout that randomly varies the parameter partitioning, while forcing them to visit as many tasks as possible at a regulated frequency, so that the network fully adapts to each update. We study the properties of our method through experiments on a variety of visual multi-task data sets. Experimental results suggest that the regularization brought by roaming has more impact on performance than usual partitioning optimization strategies. The overall method is flexible, easily applicable, provides superior regularization and consistently achieves improved performances compared to recent multi-task learning formulations.