mlsanityData to 7 Oct 2026

Meta-learning for mixed linear regression

arXiv 2002.08936 ↗First posted 20 Feb 2020cs.LG stat.MLcited by 44

Not on the map. Why?

Abstract (arXiv)

In modern supervised learning, there are a large number of tasks, but many of them are associated with only a small amount of labeled data. These include data from medical image processing and robotic interaction. Even though each individual task cannot be meaningfully trained in isolation, one seeks to meta-learn across the tasks from past experiences by exploiting some similarities. We study a fundamental question of interest: When can abundant tasks with small data compensate for lack of tasks with big data? We focus on a canonical scenario where each task is drawn from a mixture of k linear regressions, and identify sufficient conditions for such a graceful exchange to hold; The total number of examples necessary with only small data tasks scales similarly as when big data tasks are available. To this end, we introduce a novel spectral approach and show that we can efficiently utilize small data tasks with the help of tildeOmega(k3/2) medium data tasks each with tildeOmega(k1/2) examples.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 6; 2022: 11; 2023: 15; 2024: 6; 2025: 6.

44 papers.

Sort:

By area:

  1. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  2. 2023Transformers as Algorithms: Generalization and Stability in In-context Learningcited by 220
  3. 2022Provable Benefit of Multitask Representation Learning in Reinforcement Learningcited by 29
  4. 2021Sample Efficient Linear Meta-Learning by Alternating Minimizationcited by 25
  5. 2022Towards Sample-efficient Overparameterized Meta-learningcited by 21
  6. 2023Identification of Negative Transfers in Multitask Learning Using Surrogate Modelscited by 20
  7. 2022Learning Mixtures of Linear Dynamical Systemscited by 15
  8. 2023Transformers can optimally learn regression mixture modelscited by 15
  9. 2022Provable and Efficient Continual Representation Learningcited by 14
  10. 2022Subspace Recovery from Heterogeneous Data with Non-isotropic Noisecited by 14
  11. 2021A Representation Learning Perspective on the Importance of Train-Validation Splitting in Meta-Learningcited by 12
  12. 2022Learning Mixtures of Markov Chains and MDPscited by 12
32 more
  1. 2021Meta-learning with negative learning ratescited by 11
  2. 2022Understanding Benign Overfitting in Gradient-Based Meta Learningcited by 11
  3. 2023Metalearning with Very Few Samples Per Taskcited by 11
  4. 2023Tensor Decompositions Meet Control Theory: Learning General Mixtures of Linear Dynamical Systemscited by 10
  5. 2022Provable Generalization of Overparameterized Meta-learning Trained with SGDcited by 9
  6. 2022Non-stationary Bandits and Meta-Learning with a Small Set of Optimal Armscited by 8
  7. 2023Nonlinear Meta-Learning Can Guarantee Faster Ratescited by 8
  8. 2021How to distribute data across tasks for meta-learning?cited by 7
  9. 2023Towards the Theory of Unsupervised Federated Learning: Non-asymptotic Analysis of Federated EM Algorithmscited by 7
  10. 2022Global Convergence of Federated Learning for Mixed Regressioncited by 6
  11. 2022Efficient List-Decodable Regression using Batchescited by 5
  12. 2023Near-Optimal Mean Estimation with Unknown, Heteroskedastic Variancescited by 5
  13. 2023Initializing Services in Interactive ML Systems for Diverse Userscited by 5
  14. 2024Meta-Learning with Generalized Ridge Regression: High-dimensional Asymptotics, Optimality and Hyper-covariance Estimationcited by 5
  15. 2025Efficient Multivariate Robust Mean Estimation Under Mean-Shift Contaminationcited by 5
  16. 2023Provable Pathways: Learning Multiple Tasks over Multiple Pathscited by 3
  17. 2023Mixed Regression via Approximate Message Passingcited by 3
  18. 2023Linear Regression using Heterogeneous Data Batchescited by 3
  19. 2025Entangled Mean Estimation in High-Dimensionscited by 3
  20. 2025Improving Memory Efficiency for Training KANs via Meta Learningcited by 3
  21. 2025How Many Domains Suffice for Domain Generalization? A Tight Characterization via the Domain Shattering Dimensioncited by 3
  22. 2021Sample Efficient Subspace-based Representations for Nonlinear Meta-Learningcited by 2
  23. 2023Multi-dimensional domain generalization with low-rank structurescited by 2
  24. 2024Collaborative Learning with Different Labeling Functionscited by 2
  25. 2024Few-shot Multi-Task Learning of Linear Invariant Features with Meta Subspace Pursuitcited by 2
  26. 2024F-FOMAML: GNN-Enhanced Meta-Learning for Peak Period Demand Forecasting with Proxy Datacited by 1
  27. 2024In-context Learning for Mixture of Linear Regressions: Existence, Generalization and Training Dynamicscited by 1
  28. 2023Clustered Linear Contextual Bandits with Knapsackscited by 0
  29. 2023Estimation of Models with Limited Data by Leveraging Shared Structurecited by 0
  30. 2024Learning with Shared Representations: Statistical Rates and Efficient Algorithmscited by 0
  31. 2025Batch List-Decodable Linear Regression via Higher Momentscited by 0
  32. 2025On Quantification of Borrowing of Information in Hierarchical Bayesian Modelscited by 0