# HydaLearn: Highly Dynamic Task Weighting for Multi-task Learning with Auxiliary Tasks

arXiv: https://arxiv.org/abs/2008.11643 · HTML: https://mlsanity.com/paper/2008.11643/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Aug 2020.

## Built on by (most cited first, 3 of 3)

- [Learning Multi-Tasks with Inconsistent Labels by using Auxiliary Big Task](https://mlsanity.com/paper/2201.02305/) (2022)
- [Aux-NAS: Exploiting Auxiliary Labels with Negligibly Extra Inference Cost](https://mlsanity.com/paper/2405.05695/) (2024)
- [CRRS: Concentric Rectangles Regression Strategy for Multi-point Representation on Fisheye Images](https://mlsanity.com/paper/2303.14639/) (2023)

## Abstract (arXiv)

Multi-task learning (MTL) can improve performance on a task by sharing representations with one or more related auxiliary-tasks. Usually, MTL-networks are trained on a composite loss function formed by a constant weighted combination of the separate task losses. In practice, constant loss weights lead to poor results for two reasons: (i) the relevance of the auxiliary tasks can gradually drift throughout the learning process; (ii) for mini-batch based optimisation, the optimal task weights vary significantly from one update to the next depending on mini-batch sample composition. We introduce HydaLearn, an intelligent weighting algorithm that connects main-task gain to the individual task gradients, in order to inform dynamic loss weighting at the mini-batch level, addressing i and ii. Using HydaLearn, we report performance increases on synthetic data, as well as on two supervised learning domains.