# Adaptive and Robust Multi-Task Learning

arXiv: https://arxiv.org/abs/2202.05250 · HTML: https://mlsanity.com/paper/2202.05250/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 92 references that are in the map.
- First posted on arXiv 10 Feb 2022.

## Builds on (most cited first, 26 of 26)

- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [Few-Shot Learning via Learning the Representation, Provably](https://mlsanity.com/paper/2002.09434/) (2020)
- [The Benefit of Multitask Representation Learning](https://mlsanity.com/paper/1505.06279/) (2015)
- [On the Theory of Transfer Learning: The Importance of Task Diversity](https://mlsanity.com/paper/2006.11650/) (2020)
- [Provable Meta-Learning of Linear Representations](https://mlsanity.com/paper/2002.11684/) (2020)
- [Understanding and Improving Information Transfer in Multi-Task Learning](https://mlsanity.com/paper/2005.00944/) (2020)
- [Transfer Learning for High-dimensional Linear Regression: Prediction, Estimation, and Minimax Optimality](https://mlsanity.com/paper/2006.10593/) (2020)
- [The Landscape of Empirical Risk for Non-convex Losses](https://mlsanity.com/paper/1607.06534/) (2016)
- [Transfer Learning under High-dimensional Generalized Linear Models](https://mlsanity.com/paper/2105.14328/) (2021)
- [Provable Guarantees for Gradient-Based Meta-Learning](https://mlsanity.com/paper/1902.10644/) (2019)
- [Marginal Singularity, and the Benefits of Labels in Covariate-Shift](https://mlsanity.com/paper/1803.01833/) (2018)
- [Transfer Learning for Nonparametric Classification: Minimax Rate and Adaptive Classifier](https://mlsanity.com/paper/1906.02903/) (2019)
- [Learning-to-Learn Stochastic Gradient Descent with Biased Regularization](https://mlsanity.com/paper/1903.10399/) (2019)
- [Robust Covariance and Scatter Matrix Estimation under Huber's Contamination Model](https://mlsanity.com/paper/1506.00691/) (2015)
- [Near-Optimal Linear Regression under Distribution Shift](https://mlsanity.com/paper/2106.12108/) (2021)
- [A new similarity measure for covariate shift with applications to nonparametric regression](https://mlsanity.com/paper/2202.02837/) (2022)
- [Minimax Lower Bounds for Transfer Learning with Linear and One-hidden Layer Neural Networks](https://mlsanity.com/paper/2006.10581/) (2020)
- [A No-Free-Lunch Theorem for MultiTask Learning](https://mlsanity.com/paper/2006.15785/) (2020)
- [Predicting with Proxies: Transfer Learning in High Dimension](https://mlsanity.com/paper/1812.11097/) (2018)
- [Minimax Estimation for Personalized Federated Learning: An Alternative between FedAvg and Local Training?](https://mlsanity.com/paper/2103.01901/) (2021)
- [On the Sample Complexity of Adversarial Multi-Source PAC Learning](https://mlsanity.com/paper/2002.10384/) (2020)
- [New Perspectives on k-Support and Cluster Norms](https://mlsanity.com/paper/1512.08204/) (2015)
- [Minimax Rates and Adaptivity in Combining Experimental and Observational Data](https://mlsanity.com/paper/2109.10522/) (2021)
- [Data enriched linear regression](https://mlsanity.com/paper/1304.1837/) (2013)
- [Distributed Linear Model Clustering over Networks: A Tree-Based Fused-Lasso ADMM Approach](https://mlsanity.com/paper/1905.11549/) (2019)

## Built on by (most cited first, 30 of 34)

- [Learning from Similar Linear Representations: Adaptivity, Minimaxity, and Robustness](https://mlsanity.com/paper/2303.17765/) (2023)
- [Smoothness Adaptive Hypothesis Transfer Learning](https://mlsanity.com/paper/2402.14966/) (2024)
- [Optimal Multitask Linear Regression and Contextual Bandits under Sparse Heterogeneity](https://mlsanity.com/paper/2306.06291/) (2023)
- [Multi-Task Learning with Summary Statistics](https://mlsanity.com/paper/2307.02388/) (2023)
- [FedGKD: Unleashing the Power of Collaboration in Federated Graph Neural Networks](https://mlsanity.com/paper/2309.09517/) (2023)
- [Towards the Theory of Unsupervised Federated Learning: Non-asymptotic Analysis of Federated EM Algorithms](https://mlsanity.com/paper/2310.15330/) (2023)
- [Representation Retrieval Learning for Heterogeneous Data Integration](https://mlsanity.com/paper/2503.09494/) (2025)
- [Personalized Federated Learning with Multiple Known Clusters](https://mlsanity.com/paper/2204.13619/) (2022)
- [Adaptive Data Fusion for Multi-task Non-smooth Optimization](https://mlsanity.com/paper/2210.12334/) (2022)
- [Multi-dimensional domain generalization with low-rank structures](https://mlsanity.com/paper/2309.09555/) (2023)
- [Few-shot Multi-Task Learning of Linear Invariant Features with Meta Subspace Pursuit](https://mlsanity.com/paper/2409.02708/) (2024)
- [Multi-Task Dynamic Pricing in Credit Market with Contextual Information](https://mlsanity.com/paper/2410.14839/) (2024)
- [Late Fusion Multi-task Learning for Semiparametric Inference with Nuisance Parameters](https://mlsanity.com/paper/2507.07941/) (2025)
- [Joint Graph Learning and Model Fitting in Laplacian Regularized Stratified Models](https://mlsanity.com/paper/2305.02573/) (2023)
- [Learning with Shared Representations: Statistical Rates and Efficient Algorithms](https://mlsanity.com/paper/2409.04919/) (2024)
- [Understanding the Statistical Accuracy-Communication Trade-off in Personalized Federated Learning with Minimax Guarantees](https://mlsanity.com/paper/2410.08934/) (2024)
- [Sharp Gaussian approximations for Decentralized Federated Learning](https://mlsanity.com/paper/2505.08125/) (2025)
- [Measuring Fine-Grained Relatedness in Multitask Learning via Data Attribution](https://mlsanity.com/paper/2505.21438/) (2025)
- [Multi-task Learning for Heterogeneous Multi-source Block-Wise Missing Data](https://mlsanity.com/paper/2505.24413/) (2025)
- [When More Data Doesn't Help: Limits of Adaptation in Multitask Learning](https://mlsanity.com/paper/2601.20774/) (2026)
- [Personalized Multi-tier Federated Learning](https://mlsanity.com/paper/2407.14251/) (2024)
- [GeoERM: Geometry-Aware Multi-Task Representation Learning on Riemannian Manifolds](https://mlsanity.com/paper/2505.02972/) (2025)
- [Joint estimation of smooth graph signals from partial linear measurements](https://mlsanity.com/paper/2505.23240/) (2025)
- [Multi-Task Learning with Covariate-Overlap Regularization](https://mlsanity.com/paper/2505.24281/) (2025)
- [Adaptive Multi-task Learning for Multi-sector Portfolio Optimization](https://mlsanity.com/paper/2507.16433/) (2025)
- [Robust and Adaptive Spectral Method for Representation Multi-Task Learning with Contamination](https://mlsanity.com/paper/2509.06575/) (2025)
- [On the Power of Source Screening for Learning Shared Feature Extractors](https://mlsanity.com/paper/2602.16125/) (2026)
- [Adaptive Personalized Federated Learning via Multi-task Averaging of Kernel Mean Embeddings](https://mlsanity.com/paper/2603.02233/) (2026)
- [Near-optimal and Efficient First-Order Algorithm for Multi-Task Learning with Shared Linear Representation](https://mlsanity.com/paper/2605.00473/) (2026)
- [Adaptive Estimation and Inference in Semi-parametric Heterogeneous Clustered Multitask Learning via Neyman Orthogonality](https://mlsanity.com/paper/2605.01907/) (2026)

## Abstract (arXiv)

We study the multi-task learning problem that aims to simultaneously analyze multiple datasets collected from different sources and learn one model for each of them. We propose a family of adaptive methods that automatically utilize possible similarities among those tasks while carefully handling their differences. We derive sharp statistical guarantees for the methods and prove their robustness against outlier tasks. Numerical experiments on synthetic and real datasets demonstrate the efficacy of our new methods.