mlsanityData to 14 Oct 2026

When More Data Doesn't Help: Limits of Adaptation in Multitask Learning

arXiv 2601.20774 ↗First posted 28 Jan 2026cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Multitask learning and related frameworks have achieved tremendous success in modern applications. In multitask learning problem, we are given a set of heterogeneous datasets collected from related source tasks and hope to enhance the performance above what we could hope to achieve by solving each of them individually. The recent work of arXiv:2006.15785 has showed that, without access to distributional information, no algorithm based on aggregating samples alone can guarantee optimal risk as long as the sample size per task is bounded. In this paper, we focus on understanding the statistical limits of multitask learning. We go beyond the no-free-lunch theorem in arXiv:2006.15785 by establishing a stronger impossibility result of adaptation that holds for arbitrarily large sample size per task. This improvement conveys an important message that the hardness of multitask learning cannot be overcame by having abundant data per task. We also discuss the notion of optimal adaptivity that may be of future interests.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 29 of its 52 references that are in the map. First posted on arXiv 28 Jan 2026.

Builds on

29 of 52 references.

References · 29

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  2. 2019Invariant Risk Minimizationcited by 1,854
  3. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 711
  4. 2013Domain Generalization via Invariant Feature Representationcited by 623
  5. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  6. 2020Few-Shot Learning via Learning the Representation, Provablycited by 214
  7. 2015The Benefit of Multitask Representation Learningcited by 195
  8. 2020On the Theory of Transfer Learning: The Importance of Task Diversitycited by 186
  9. 2020Provable Meta-Learning of Linear Representationscited by 161
  10. 2019Provable Guarantees for Gradient-Based Meta-Learningcited by 67
  11. 2020On the Value of Target Data in Transfer Learningcited by 65
  12. 2019Learning-to-Learn Stochastic Gradient Descent with Biased Regularizationcited by 49
17 more
  1. 2015Fast rates in statistical and online learningcited by 47
  2. 2022Adaptive and Robust Multi-Task Learningcited by 34
  3. 2022MAML and ANIL Provably Learn Representationscited by 30
  4. 2022Byzantine-Robust Federated Learning with Optimal Statistical Rates and Privacy Guaranteescited by 30
  5. 2020A No-Free-Lunch Theorem for MultiTask Learningcited by 28
  6. 2014Efficient Representations for Life-Long Learning and Autoencodingcited by 23
  7. 2022Robust angle-based transfer learning in high dimensionscited by 16
  8. 2017Learning Discrete Distributions from Untrusted Batchescited by 14
  9. 2022Subspace Recovery from Heterogeneous Data with Non-isotropic Noisecited by 14
  10. 2020On the Sample Complexity of Adversarial Multi-Source PAC Learningcited by 11
  11. 2023Metalearning with Very Few Samples Per Taskcited by 11
  12. 2023Limits of Model Selection under Transfer Learningcited by 9
  13. 2018Do Outliers Ruin Collaboration?cited by 7
  14. 2023Efficient Estimation of the Central Mean Subspace via Smoothed Gradient Outer Productscited by 7
  15. 2022Multitask Learning via Shared Features: Algorithms and Hardnesscited by 5
  16. 2024Help Me Help You: The Aggregate Value of Source and Target Data in Transfer Learningcited by 4
  17. 2025Transfer Neyman-Pearson Algorithm for Outlier Detectioncited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Optimal Learning Under Tsybakov Noisecited by 0