# Auxiliary Task Reweighting for Minimum-data Learning

arXiv: https://arxiv.org/abs/2010.08244 · HTML: https://mlsanity.com/paper/2010.08244/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Oct 2020.

## Built on by (most cited first, 23 of 23)

- [Conflict-Averse Gradient Descent for Multi-task Learning](https://mlsanity.com/paper/2110.14048/) (2021)
- [ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learning](https://mlsanity.com/paper/2301.12618/) (2023)
- [Optimization with Access to Auxiliary Information](https://mlsanity.com/paper/2206.00395/) (2022)
- [AANG: Automating Auxiliary Learning](https://mlsanity.com/paper/2205.14082/) (2022)
- [Auxiliary Learning as an Asymmetric Bargaining Game](https://mlsanity.com/paper/2301.13501/) (2023)
- [Unified Convergence Theory of Stochastic and Variance-Reduced Cubic Newton Methods](https://mlsanity.com/paper/2302.11962/) (2023)
- [Linear Speedup in Personalized Collaborative Learning](https://mlsanity.com/paper/2111.05968/) (2021)
- [Adaptive Mixing of Auxiliary Losses in Supervised Learning](https://mlsanity.com/paper/2202.03250/) (2022)
- [Understanding Knowledge Transferability for Transfer Learning: A Survey](https://mlsanity.com/paper/2507.03175/) (2025)
- [Auxiliary Learning for Self-Supervised Video Representation via Similarity-based Knowledge Distillation](https://mlsanity.com/paper/2112.04011/) (2021)
- [Multiple Instance Learning with Auxiliary Task Weighting for Multiple Myeloma Classification](https://mlsanity.com/paper/2107.07805/) (2021)
- [D4AM: A General Denoising Framework for Downstream Acoustic Models](https://mlsanity.com/paper/2311.16595/) (2023)
- [Enhancing Molecular Property Prediction with Auxiliary Learning and Task-Specific Adaptation](https://mlsanity.com/paper/2401.16299/) (2024)
- [Aux-NAS: Exploiting Auxiliary Labels with Negligibly Extra Inference Cost](https://mlsanity.com/paper/2405.05695/) (2024)
- [Generalized Task-Driven Medical Image Quality Enhancement with Gradient Promotion](https://mlsanity.com/paper/2501.01114/) (2025)
- [MAC: A Conversion Rate Prediction Benchmark Featuring Labels Under Multiple Attribution Mechanisms](https://mlsanity.com/paper/2603.02184/) (2026)
- [Hyperspectral Image Super-Resolution with Spectral Mixup and Heterogeneous Datasets](https://mlsanity.com/paper/2101.07589/) (2021)
- [Auxiliary Loss Reweighting for Image Inpainting](https://mlsanity.com/paper/2111.07279/) (2021)
- [Two-Stage Multi-task Self-Supervised Learning for Medical Image Segmentation](https://mlsanity.com/paper/2402.07119/) (2024)
- [Adept: Annotation-Denoising Auxiliary Tasks with Discrete Cosine Transform Map and Keypoint for Human-Centric Pretraining](https://mlsanity.com/paper/2504.20800/) (2025)
- [Test-Time Warmup for Multimodal Large Language Models](https://mlsanity.com/paper/2509.10641/) (2025)
- [Semi-overlapping Multi-bandit Best Arm Identification for Sequential Support Network Learning](https://mlsanity.com/paper/2512.24959/) (2025)
- [PC-MCL: Patient-Consistent Multi-Cycle Learning with multi-label bias correction for respiratory sound classification](https://mlsanity.com/paper/2601.17080/) (2026)

## Abstract (arXiv)

Supervised learning requires a large amount of training data, limiting its application where labeled data is scarce. To compensate for data scarcity, one possible method is to utilize auxiliary tasks to provide additional supervision for the main task. Assigning and optimizing the importance weights for different auxiliary tasks remains an crucial and largely understudied research question. In this work, we propose a method to automatically reweight auxiliary tasks in order to reduce the data requirement on the main task. Specifically, we formulate the weighted likelihood function of auxiliary tasks as a surrogate prior for the main task. By adjusting the auxiliary task weights to minimize the divergence between the surrogate prior and the true prior of the main task, we obtain a more accurate prior estimation, achieving the goal of minimizing the required amount of training data for the main task and avoiding a costly grid search. In multiple experimental settings (e.g. semi-supervised learning, multi-label classification), we demonstrate that our algorithm can effectively utilize limited labeled data of the main task with the benefit of auxiliary tasks compared with previous task reweighting methods. We also show that under extreme cases with only a few extra examples (e.g. few-shot domain adaptation), our algorithm results in significant improvement over the baseline.