# Adaptive Scheduling for Multi-Task Learning

arXiv: https://arxiv.org/abs/1909.06434 · HTML: https://mlsanity.com/paper/1909.06434/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Sep 2019.

## Built on by (most cited first, 24 of 24)

- [UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretraining](https://mlsanity.com/paper/2304.09151/) (2023)
- [FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion Tasks](https://mlsanity.com/paper/2303.02483/) (2023)
- [MFTCoder: Boosting Code LLMs with Multitask Fine-Tuning](https://mlsanity.com/paper/2311.02303/) (2023)
- [Causes and Cures for Interference in Multilingual Translation](https://mlsanity.com/paper/2212.07530/) (2022)
- [Robust Optimization for Multilingual Translation with Imbalanced Data](https://mlsanity.com/paper/2104.07639/) (2021)
- [Value-Consistent Representation Learning for Data-Efficient Reinforcement Learning](https://mlsanity.com/paper/2206.12542/) (2022)
- [Learning Temporally-Consistent Representations for Data-Efficient Reinforcement Learning](https://mlsanity.com/paper/2110.04935/) (2021)
- [Balancing Average and Worst-case Accuracy in Multitask Learning](https://mlsanity.com/paper/2110.05838/) (2021)
- [Training multi-objective/multi-task collocation physics-informed neural network with student/teachers transfer learnings](https://mlsanity.com/paper/2107.11496/) (2021)
- [Competence-based Curriculum Learning for Multilingual Machine Translation](https://mlsanity.com/paper/2109.04002/) (2021)
- [Let the Model Decide its Curriculum for Multitask Learning](https://mlsanity.com/paper/2205.09898/) (2022)
- [Order Matters in the Presence of Dataset Imbalance for Multilingual Learning](https://mlsanity.com/paper/2312.06134/) (2023)
- [PLMCL: Partial-Label Momentum Curriculum Learning for Multi-Label Image Classification](https://mlsanity.com/paper/2208.09999/) (2022)
- [Detailed delineation of the fetal brain in diffusion MRI via multi-task learning](https://mlsanity.com/paper/2409.06716/) (2024)
- [Challenges and Opportunities of Using Transformer-Based Multi-Task Learning in NLP Through ML Lifecycle: A Survey](https://mlsanity.com/paper/2308.08234/) (2023)
- [Bidirectional Progressive Neural Networks with Episodic Return Progress for Emergent Task Sequencing and Robotic Skill Transfer](https://mlsanity.com/paper/2403.04001/) (2024)
- [MTLComb: multi-task learning combining regression and classification tasks for joint feature selection](https://mlsanity.com/paper/2405.09886/) (2024)
- [Can the Variation of Model Weights be used as a Criterion for Self-Paced Multilingual NMT?](https://mlsanity.com/paper/2410.04147/) (2024)
- [Interactive LLM-assisted Curriculum Learning for Multi-Task Evolutionary Policy Search](https://mlsanity.com/paper/2602.10891/) (2026)
- [A general approach to enhance the survivability of backdoor attacks by decision path coupling](https://mlsanity.com/paper/2403.02950/) (2024)
- [Optimizing the Training Schedule of Multilingual NMT using Reinforcement Learning](https://mlsanity.com/paper/2410.06118/) (2024)
- [Deploying Multi-task Online Server with Large Language Model](https://mlsanity.com/paper/2411.03644/) (2024)
- [LLaVA-NeuMT: Selective Layer-Neuron Modulation for Efficient Multilingual Multimodal Translation](https://mlsanity.com/paper/2507.18940/) (2025)
- [Using Temperature Sampling to Effectively Train Robot Learning Policies on Imbalanced Datasets](https://mlsanity.com/paper/2510.19373/) (2025)

## Abstract (arXiv)

To train neural machine translation models simultaneously on multiple tasks (languages), it is common to sample each task uniformly or in proportion to dataset sizes. As these methods offer little control over performance trade-offs, we explore different task scheduling approaches. We first consider existing non-adaptive techniques, then move on to adaptive schedules that over-sample tasks with poorer results compared to their respective baseline. As explicit schedules can be inefficient, especially if one task is highly over-sampled, we also consider implicit schedules, learning to scale learning rates or gradients of individual tasks instead. These techniques allow training multilingual models that perform better for low-resource language pairs (tasks with small amount of data), while minimizing negative effects on high-resource tasks.