# A Simple General Approach to Balance Task Difficulty in Multi-Task Learning

arXiv: https://arxiv.org/abs/2002.04792 · HTML: https://mlsanity.com/paper/2002.04792/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Feb 2020.

## Built on by (most cited first, 8 of 8)

- [Model-Augmented Q-learning](https://mlsanity.com/paper/2102.03866/) (2021)
- [Quantum-Enhanced Multi-Task Learning with Learnable Weighting for Pharmacokinetic and Toxicity Prediction](https://mlsanity.com/paper/2509.04601/) (2025)
- [Graph Coloring for Multi-Task Learning](https://mlsanity.com/paper/2509.16959/) (2025)
- [STEPER: Step-wise Knowledge Distillation for Enhancing Reasoning Ability in Multi-Step Retrieval-Augmented Language Models](https://mlsanity.com/paper/2510.07923/) (2025)
- [Simple and Effective Balance of Contrastive Losses](https://mlsanity.com/paper/2112.11743/) (2021)
- [Robust Hierarchical Patterns for identifying MDD patients: A Multisite Study](https://mlsanity.com/paper/2202.11144/) (2022)
- [Learning Easily Updated General Purpose Text Representations with Adaptable Task-Specific Prefixes](https://mlsanity.com/paper/2305.13499/) (2023)
- [Neuro-Visualizer: An Auto-encoder-based Loss Landscape Visualization Method](https://mlsanity.com/paper/2309.14601/) (2023)

## Abstract (arXiv)

In multi-task learning, difficulty levels of different tasks are varying. There are many works to handle this situation and we classify them into five categories, including the direct sum approach, the weighted sum approach, the maximum approach, the curriculum learning approach, and the multi-objective optimization approach. Those approaches have their own limitations, for example, using manually designed rules to update task weights, non-smooth objective function, and failing to incorporate other functions than training losses. In this paper, to alleviate those limitations, we propose a Balanced Multi-Task Learning (BMTL) framework. Different from existing studies which rely on task weighting, the BMTL framework proposes to transform the training loss of each task to balance difficulty levels among tasks based on an intuitive idea that tasks with larger training losses will receive more attention during the optimization procedure. We analyze the transformation function and derive necessary conditions. The proposed BMTL framework is very simple and it can be combined with most multi-task learning models. Empirical studies show the state-of-the-art performance of the proposed BMTL framework.