# Identifying beneficial task relations for multi-task learning in deep neural networks

arXiv: https://arxiv.org/abs/1702.08303 · HTML: https://mlsanity.com/paper/1702.08303/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Feb 2017.

## Built on by (most cited first, 30 of 59)

- [SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer](https://mlsanity.com/paper/2110.07904/) (2021)
- [ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning](https://mlsanity.com/paper/2111.10952/) (2021)
- [What to Pre-Train on? Efficient Intermediate Task Selection](https://mlsanity.com/paper/2104.08247/) (2021)
- [PIGEON: Predicting Image Geolocations](https://mlsanity.com/paper/2307.05845/) (2023)
- [A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning](https://mlsanity.com/paper/2408.07057/) (2024)
- [π-Tuning: Transferring Multimodal Foundation Models with Optimal Multi-task Interpolation](https://mlsanity.com/paper/2304.14381/) (2023)
- [Towards Versatile Embodied Navigation](https://mlsanity.com/paper/2210.16822/) (2022)
- [Revisit Multimodal Meta-Learning through the Lens of Multi-Task Learning](https://mlsanity.com/paper/2110.14202/) (2021)
- [Pareto Set Learning for Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2501.06773/) (2025)
- [Task Compass: Scaling Multi-task Pre-training with Task Prefix](https://mlsanity.com/paper/2210.06277/) (2022)
- [To Share or not to Share: Predicting Sets of Sources for Model Transfer Learning](https://mlsanity.com/paper/2104.08078/) (2021)
- [Robust Natural Language Processing: Recent Advances, Challenges, and Future Directions](https://mlsanity.com/paper/2201.00768/) (2022)
- [FETA: A Benchmark for Few-Sample Task Transfer in Open-Domain Dialogue](https://mlsanity.com/paper/2205.06262/) (2022)
- [Language Modelling as a Multi-Task Problem](https://mlsanity.com/paper/2101.11287/) (2021)
- [Contrastive Modules with Temporal Attention for Multi-Task Reinforcement Learning](https://mlsanity.com/paper/2311.01075/) (2023)
- [Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks](https://mlsanity.com/paper/2301.05065/) (2023)
- [Interpreting and Exploiting Functional Specialization in Multi-Head Attention under Multi-task Learning](https://mlsanity.com/paper/2310.10318/) (2023)
- [Grad2Task: Improved Few-shot Text Classification Using Gradients for Task Representation](https://mlsanity.com/paper/2201.11576/) (2022)
- [Let the Model Decide its Curriculum for Multitask Learning](https://mlsanity.com/paper/2205.09898/) (2022)
- [Multi-Task Meta Learning: learn how to adapt to unseen tasks](https://mlsanity.com/paper/2210.06989/) (2022)
- [GradTS: A Gradient-Based Automatic Auxiliary Task Selection Method Based on Transformer Networks](https://mlsanity.com/paper/2109.05748/) (2021)
- [Efficiently Tuned Parameters are Task Embeddings](https://mlsanity.com/paper/2210.11705/) (2022)
- [Latent-Optimized Adversarial Neural Transfer for Sarcasm Detection](https://mlsanity.com/paper/2104.09261/) (2021)
- [When Multi-Task Learning Meets Partial Supervision: A Computer Vision Review](https://mlsanity.com/paper/2307.14382/) (2023)
- [Towards Better Multi-task Learning: A Framework for Optimizing Dataset Combinations in Large Language Models](https://mlsanity.com/paper/2412.11455/) (2024)
- [Multi-task Learning by Leveraging the Semantic Information](https://mlsanity.com/paper/2103.02546/) (2021)
- [Constructing Time-Series Momentum Portfolios with Deep Multi-Task Learning](https://mlsanity.com/paper/2306.13661/) (2023)
- [Learning More for Free - A Multi Task Learning Approach for Improved Pathology Classification in Capsule Endoscopy](https://mlsanity.com/paper/2106.16162/) (2021)
- [A Flexible Multi-Task Model for BERT Serving](https://mlsanity.com/paper/2107.05377/) (2021)
- [MT-GBM: A Multi-Task Gradient Boosting Machine with Shared Decision Trees](https://mlsanity.com/paper/2201.06239/) (2022)

## Abstract (arXiv)

Multi-task learning (MTL) in deep neural networks for NLP has recently received increasing interest due to some compelling benefits, including its potential to efficiently regularize models and to reduce the need for labeled data. While it has brought significant improvements in a number of NLP tasks, mixed results have been reported, and little is known about the conditions under which MTL leads to gains in NLP. This paper sheds light on the specific task relations that can lead to gains from MTL models over single-task setups.