# When More Data Doesn't Help: Limits of Adaptation in Multitask Learning

arXiv: https://arxiv.org/abs/2601.20774 · HTML: https://mlsanity.com/paper/2601.20774/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 52 references that are in the map.
- First posted on arXiv 28 Jan 2026.

## Builds on (most cited first, 29 of 29)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates](https://mlsanity.com/paper/1803.01498/) (2018)
- [Domain Generalization via Invariant Feature Representation](https://mlsanity.com/paper/1301.2115/) (2013)
- [Exploiting Shared Representations for Personalized Federated Learning](https://mlsanity.com/paper/2102.07078/) (2021)
- [Few-Shot Learning via Learning the Representation, Provably](https://mlsanity.com/paper/2002.09434/) (2020)
- [The Benefit of Multitask Representation Learning](https://mlsanity.com/paper/1505.06279/) (2015)
- [On the Theory of Transfer Learning: The Importance of Task Diversity](https://mlsanity.com/paper/2006.11650/) (2020)
- [Provable Meta-Learning of Linear Representations](https://mlsanity.com/paper/2002.11684/) (2020)
- [Provable Guarantees for Gradient-Based Meta-Learning](https://mlsanity.com/paper/1902.10644/) (2019)
- [On the Value of Target Data in Transfer Learning](https://mlsanity.com/paper/2002.04747/) (2020)
- [Learning-to-Learn Stochastic Gradient Descent with Biased Regularization](https://mlsanity.com/paper/1903.10399/) (2019)
- [Fast rates in statistical and online learning](https://mlsanity.com/paper/1507.02592/) (2015)
- [Adaptive and Robust Multi-Task Learning](https://mlsanity.com/paper/2202.05250/) (2022)
- [MAML and ANIL Provably Learn Representations](https://mlsanity.com/paper/2202.03483/) (2022)
- [Byzantine-Robust Federated Learning with Optimal Statistical Rates and Privacy Guarantees](https://mlsanity.com/paper/2205.11765/) (2022)
- [A No-Free-Lunch Theorem for MultiTask Learning](https://mlsanity.com/paper/2006.15785/) (2020)
- [Efficient Representations for Life-Long Learning and Autoencoding](https://mlsanity.com/paper/1411.1490/) (2014)
- [Robust angle-based transfer learning in high dimensions](https://mlsanity.com/paper/2210.12759/) (2022)
- [Learning Discrete Distributions from Untrusted Batches](https://mlsanity.com/paper/1711.08113/) (2017)
- [Subspace Recovery from Heterogeneous Data with Non-isotropic Noise](https://mlsanity.com/paper/2210.13497/) (2022)
- [On the Sample Complexity of Adversarial Multi-Source PAC Learning](https://mlsanity.com/paper/2002.10384/) (2020)
- [Metalearning with Very Few Samples Per Task](https://mlsanity.com/paper/2312.13978/) (2023)
- [Limits of Model Selection under Transfer Learning](https://mlsanity.com/paper/2305.00152/) (2023)
- [Do Outliers Ruin Collaboration?](https://mlsanity.com/paper/1805.04720/) (2018)
- [Efficient Estimation of the Central Mean Subspace via Smoothed Gradient Outer Products](https://mlsanity.com/paper/2312.15469/) (2023)
- [Multitask Learning via Shared Features: Algorithms and Hardness](https://mlsanity.com/paper/2209.03112/) (2022)
- [Help Me Help You: The Aggregate Value of Source and Target Data in Transfer Learning](https://mlsanity.com/paper/2408.16189/) (2024)
- [Transfer Neyman-Pearson Algorithm for Outlier Detection](https://mlsanity.com/paper/2501.01525/) (2025)

## Built on by (most cited first, 1 of 1)

- [Optimal Learning Under Tsybakov Noise](https://mlsanity.com/paper/2608.08416/) (2026)

## Abstract (arXiv)

Multitask learning and related frameworks have achieved tremendous success in modern applications. In multitask learning problem, we are given a set of heterogeneous datasets collected from related source tasks and hope to enhance the performance above what we could hope to achieve by solving each of them individually. The recent work of arXiv:2006.15785 has showed that, without access to distributional information, no algorithm based on aggregating samples alone can guarantee optimal risk as long as the sample size per task is bounded. In this paper, we focus on understanding the statistical limits of multitask learning. We go beyond the no-free-lunch theorem in arXiv:2006.15785 by establishing a stronger impossibility result of adaptation that holds for arbitrarily large sample size per task. This improvement conveys an important message that the hardness of multitask learning cannot be overcame by having abundant data per task. We also discuss the notion of optimal adaptivity that may be of future interests.