# Many Task Learning with Task Routing

arXiv: https://arxiv.org/abs/1903.12117 · HTML: https://mlsanity.com/paper/1903.12117/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Mar 2019.

## Built on by (most cited first, 30 of 50)

- [UniT: Multimodal Multitask Learning with a Unified Transformer](https://mlsanity.com/paper/2102.10772/) (2021)
- [InstructIR: High-Quality Image Restoration Following Human Instructions](https://mlsanity.com/paper/2401.16468/) (2024)
- [Modular Deep Learning](https://mlsanity.com/paper/2302.11529/) (2023)
- [Towards Modular LLMs by Building and Reusing a Library of LoRAs](https://mlsanity.com/paper/2405.11157/) (2024)
- [Distribution Matching for Multi-Task Learning of Classification Tasks: a Large-Scale Study on Faces & Beyond](https://mlsanity.com/paper/2401.01219/) (2024)
- [Uni-Perceiver-MoE: Learning Sparse Generalist Models with Conditional MoEs](https://mlsanity.com/paper/2206.04674/) (2022)
- [MulT: An End-to-End Multitask Learning Transformer](https://mlsanity.com/paper/2205.08303/) (2022)
- [One Model is All You Need: Multi-Task Learning Enables Simultaneous Histology Image Segmentation and Classification](https://mlsanity.com/paper/2203.00077/) (2022)
- [Multimodal Instruction Tuning with Conditional Mixture of LoRA](https://mlsanity.com/paper/2402.15896/) (2024)
- [Multi-Task Learning with Multi-Query Transformer for Dense Prediction](https://mlsanity.com/paper/2205.14354/) (2022)
- [INTERN: A New Learning Paradigm Towards General Vision](https://mlsanity.com/paper/2111.08687/) (2021)
- [Denoising Task Routing for Diffusion Models](https://mlsanity.com/paper/2310.07138/) (2023)
- [Towards Versatile Embodied Navigation](https://mlsanity.com/paper/2210.16822/) (2022)
- [Universal Representations: A Unified Look at Multiple Task and Domain Learning](https://mlsanity.com/paper/2204.02744/) (2022)
- [Scalarization for Multi-Task and Multi-Domain Learning at Scale](https://mlsanity.com/paper/2310.08910/) (2023)
- [Mitigating Task Interference in Multi-Task Learning via Explicit Task Routing with Non-Learnable Primitives](https://mlsanity.com/paper/2308.02066/) (2023)
- [MT2ST: Adaptive Multi-Task to Single-Task Learning](https://mlsanity.com/paper/2406.18038/) (2024)
- [Variational Multi-Task Learning with Gumbel-Softmax Priors](https://mlsanity.com/paper/2111.05323/) (2021)
- [Towards a Unified Foundation Model: Jointly Pre-Training Transformers on Unpaired Images and Text](https://mlsanity.com/paper/2112.07074/) (2021)
- [Supervised Fine-tuning in turn Improves Visual Foundation Models](https://mlsanity.com/paper/2401.10222/) (2024)
- [Same Same, But Different: Conditional Multi-Task Learning for Demographic-Specific Toxicity Detection](https://mlsanity.com/paper/2302.07372/) (2023)
- [Multi-task Learning with 3D-Aware Regularization](https://mlsanity.com/paper/2310.00986/) (2023)
- [Episodic Multi-Task Learning with Heterogeneous Neural Processes](https://mlsanity.com/paper/2310.18713/) (2023)
- [MINOTAUR: Multi-task Video Grounding From Multimodal Queries](https://mlsanity.com/paper/2302.08063/) (2023)
- [Instance-Level Task Parameters: A Robust Multi-task Weighting Framework](https://mlsanity.com/paper/2106.06129/) (2021)
- [Improved optimization strategies for deep Multi-Task Networks](https://mlsanity.com/paper/2109.11678/) (2021)
- [Learning More May Not Be Better: Knowledge Transferability in Vision and Language Tasks](https://mlsanity.com/paper/2208.10758/) (2022)
- [Provable Pathways: Learning Multiple Tasks over Multiple Paths](https://mlsanity.com/paper/2303.04338/) (2023)
- [Grounding Degradations in Natural Language for All-In-One Video Restoration](https://mlsanity.com/paper/2507.14851/) (2025)
- [Revisit the Imbalance Optimization in Multi-task Learning: An Experimental Analysis](https://mlsanity.com/paper/2509.23915/) (2025)

## Abstract (arXiv)

Typical multi-task learning (MTL) methods rely on architectural adjustments and a large trainable parameter set to jointly optimize over several tasks. However, when the number of tasks increases so do the complexity of the architectural adjustments and resource requirements. In this paper, we introduce a method which applies a conditional feature-wise transformation over the convolutional activations that enables a model to successfully perform a large number of tasks. To distinguish from regular MTL, we introduce Many Task Learning (MaTL) as a special case of MTL where more than 20 tasks are performed by a single model. Our method dubbed Task Routing (TR) is encapsulated in a layer we call the Task Routing Layer (TRL), which applied in an MaTL scenario successfully fits hundreds of classification tasks in one model. We evaluate our method on 5 datasets against strong baselines and state-of-the-art approaches.