# Interpetable Target-Feature Aggregation for Multi-Task Learning based on Bias-Variance Analysis

arXiv: https://arxiv.org/abs/2406.07991 · HTML: https://mlsanity.com/paper/2406.07991/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 34 references that are in the map.
- First posted on arXiv 12 Jun 2024.

## Builds on (most cited first, 12 of 12)

- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [MultiMAE: Multi-modal Multi-task Masked Autoencoders](https://mlsanity.com/paper/2204.01678/) (2022)
- [Efficiently Identifying Task Groupings for Multi-Task Learning](https://mlsanity.com/paper/2109.04617/) (2021)
- [Multi-Task Learning as a Bargaining Game](https://mlsanity.com/paper/2202.01017/) (2022)
- [BERT for Joint Intent Classification and Slot Filling](https://mlsanity.com/paper/1902.10909/) (2019)
- [Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learning](https://mlsanity.com/paper/2111.10603/) (2021)
- [LibMTL: A Python Library for Multi-Task Learning](https://mlsanity.com/paper/2203.14338/) (2022)
- [Survey of explainable machine learning with visual and granular methods beyond quasi-explanations](https://mlsanity.com/paper/2009.10221/) (2020)
- [Learnable Filters for Geometric Scattering Modules](https://mlsanity.com/paper/2208.07458/) (2022)
- [Multi-Task Learning with Summary Statistics](https://mlsanity.com/paper/2307.02388/) (2023)
- [Interpretable Linear Dimensionality Reduction based on Bias-Variance Analysis](https://mlsanity.com/paper/2303.14734/) (2023)
- [Nonlinear Feature Aggregation: Two Algorithms driven by Theory](https://mlsanity.com/paper/2306.11143/) (2023)

## Abstract (arXiv)

Multi-task learning (MTL) is a powerful machine learning paradigm designed to leverage shared knowledge across tasks to improve generalization and performance. Previous works have proposed approaches to MTL that can be divided into feature learning, focused on the identification of a common feature representation, and task clustering, where similar tasks are grouped together. In this paper, we propose an MTL approach at the intersection between task clustering and feature transformation based on a two-phase iterative aggregation of targets and features. First, we propose a bias-variance analysis for regression models with additive Gaussian noise, where we provide a general expression of the asymptotic bias and variance of a task, considering a linear regression trained on aggregated input features and an aggregated target. Then, we exploit this analysis to provide a two-phase MTL algorithm (NonLinCTFA). Firstly, this method partitions the tasks into clusters and aggregates each obtained group of targets with their mean. Then, for each aggregated task, it aggregates subsets of features with their mean in a dimensionality reduction fashion. In both phases, a key aspect is to preserve the interpretability of the reduced targets and features through the aggregation with the mean, which is further motivated by applications to Earth science. Finally, we validate the algorithms on synthetic data, showing the effect of different parameters and real-world datasets, exploring the validity of the proposed methodology on classical datasets, recent baselines, and Earth science applications.