# Leveraging convergence behavior to balance conflicting tasks in multi-task learning

arXiv: https://arxiv.org/abs/2204.06698 · HTML: https://mlsanity.com/paper/2204.06698/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 34 references that are in the map.
- First posted on arXiv 14 Apr 2022.

## Builds on (most cited first, 18 of 18)

- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [Multi-Task Learning as Multi-Objective Optimization](https://mlsanity.com/paper/1810.04650/) (2018)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [End-to-End Multi-Task Learning with Attention](https://mlsanity.com/paper/1803.10704/) (2018)
- [Cross-stitch Networks for Multi-task Learning](https://mlsanity.com/paper/1604.03539/) (2016)
- [Pareto Multi-Task Learning](https://mlsanity.com/paper/1912.12854/) (2019)
- [Recurrent Neural Network for Text Classification with Multi-Task Learning](https://mlsanity.com/paper/1605.05101/) (2016)
- [Adversarial Multi-task Learning for Text Classification](https://mlsanity.com/paper/1704.05742/) (2017)
- [MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving](https://mlsanity.com/paper/1612.07695/) (2016)
- [Benchmark Analysis of Representative Deep Neural Network Architectures](https://mlsanity.com/paper/1810.00736/) (2018)
- [Branched Multi-Task Networks: Deciding What Layers To Share](https://mlsanity.com/paper/1904.02920/) (2019)
- [AuxNet: Auxiliary tasks enhanced Semantic Segmentation for Automated Driving](https://mlsanity.com/paper/1901.05808/) (2019)
- [Self-Paced Multi-Task Learning](https://mlsanity.com/paper/1604.01474/) (2016)
- [Pixel-level Encoding and Depth Layering for Instance-level Semantic Labeling](https://mlsanity.com/paper/1604.05096/) (2016)
- [Follow the bisector: a simple method for multi-objective optimization](https://mlsanity.com/paper/2007.06937/) (2020)
- [Multi-task human analysis in still images: 2D/3D pose, depth map, and multi-part segmentation](https://mlsanity.com/paper/1905.03003/) (2019)

## Built on by (most cited first, 2 of 2)

- [Semantic SuperPoint: A Deep Semantic Descriptor](https://mlsanity.com/paper/2211.01098/) (2022)
- [Uniform Loss vs. Specialized Optimization: A Comparative Analysis in Multi-Task Learning](https://mlsanity.com/paper/2505.10347/) (2025)

## Abstract (arXiv)

Multi-Task Learning is a learning paradigm that uses correlated tasks to improve performance generalization. A common way to learn multiple tasks is through the hard parameter sharing approach, in which a single architecture is used to share the same subset of parameters, creating an inductive bias between them during the training process. Due to its simplicity, potential to improve generalization, and reduce computational cost, it has gained the attention of the scientific and industrial communities. However, tasks often conflict with each other, which makes it challenging to define how the gradients of multiple tasks should be combined to allow simultaneous learning. To address this problem, we use the idea of multi-objective optimization to propose a method that takes into account temporal behaviour of the gradients to create a dynamic bias that adjust the importance of each task during the backpropagation. The result of this method is to give more attention to the tasks that are diverging or that are not being benefited during the last iterations, allowing to ensure that the simultaneous learning is heading to the performance maximization of all tasks. As a result, we empirically show that the proposed method outperforms the state-of-art approaches on learning conflicting tasks. Unlike the adopted baselines, our method ensures that all tasks reach good generalization performances.