mlsanityData to 14 Oct 2026

Robust and Adaptive Spectral Method for Representation Multi-Task Learning with Contamination

arXiv 2509.06575 ↗First posted 8 Sep 2025stat.ML cs.LG stat.MEcited by 0

Not on the map. Why?

Abstract (arXiv)

Representation-based multi-task learning (MTL) improves efficiency by learning a shared structure across tasks, but its practical application is often hindered by contamination, outliers, or adversarial tasks. Most existing methods and theories assume a clean or near-clean setting, failing when contamination is significant. This paper tackles representation MTL with an unknown and potentially large contamination proportion, while also allowing for heterogeneity among inlier tasks. We introduce a Robust and Adaptive Spectral method (RAS) that can distill the shared inlier representation effectively and efficiently, while requiring no prior knowledge of the contamination level or the true representation dimension. Theoretically, we provide non-asymptotic error bounds for both the learned representation and the per-task parameters. These bounds adapt to inlier task similarity and outlier structure, and guarantee that RAS performs at least as well as single-task learning, thus preventing negative transfer. We also extend our framework to transfer learning with corresponding theoretical guarantees for the target task. Extensive experiments confirm our theory, showcasing the robustness and adaptivity of RAS, and its superior performance in regimes with up to 80% task contamination.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 70 references that are in the map. First posted on arXiv 8 Sep 2025.

Builds on

27 of 70 references.

References · 27

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
  3. 2019A Comprehensive Survey on Transfer Learningcited by 1,160
  4. 2017A Survey on Multi-Task Learningcited by 840
  5. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
  6. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 803
  7. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 711
  8. 2017Federated Multi-Task Learningcited by 672
  9. 2013Domain Generalization via Invariant Feature Representationcited by 623
  10. 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 375
  11. 2020Multi-Task Learning with Deep Neural Networks: A Surveycited by 375
  12. 2016DeepSurv: Personalized Treatment Recommender System Using A Cox Proportional Hazards Deep Neural Networkcited by 358
15 more
  1. 2018Meta-Learning: A Surveycited by 256
  2. 2020Few-Shot Learning via Learning the Representation, Provablycited by 214
  3. 2015The Benefit of Multitask Representation Learningcited by 195
  4. 2020On the Theory of Transfer Learning: The Importance of Task Diversitycited by 186
  5. 2020Provable Meta-Learning of Linear Representationscited by 161
  6. 2019Mean estimation and regression under heavy-tailed distributions--a surveycited by 136
  7. 2020Spectral Methods for Data Science: A Statistical Perspectivecited by 112
  8. 2016Deep Survival Analysiscited by 61
  9. 2014Non-convex Robust PCAcited by 41
  10. 2021How Fine-Tuning Allows for Effective Meta-Learningcited by 41
  11. 2022Adaptive and Robust Multi-Task Learningcited by 34
  12. 2023Learning from Similar Linear Representations: Adaptivity, Minimaxity, and Robustnesscited by 28
  13. 2022Robust angle-based transfer learning in high dimensionscited by 16
  14. 2022Subspace Recovery from Heterogeneous Data with Non-isotropic Noisecited by 14
  15. 2021A Representation Learning Perspective on the Importance of Train-Validation Splitting in Meta-Learningcited by 12

Built on by

Not cited yet.