mlsanityData to 14 Oct 2026

The Benefit of Multitask Representation Learning

arXiv 1505.06279 ↗First posted 23 May 2015stat.ML cs.LGcited by 195

Not on the map. Why?

Abstract (arXiv)

We discuss a general method to learn data representations from multiple tasks. We provide a justification for this method in both settings of multitask learning and learning-to-learn. The method is illustrated in detail in the special case of linear feature learning. Conditions on the theoretical advantage offered by multitask representation learning over independent task learning are established. In particular, focusing on the important example of half-space learning, we derive the regime in which multitask representation learning is beneficial over independent task learning, as a function of the sample size, the number of tasks and the intrinsic data dimensionality. Other potential applications of our results include multitask feature learning in reproducing kernel Hilbert spaces and multilayer, deep networks.

Key facts. Cited by 195 papers in the map (data through 14 Oct 2026). First posted on arXiv 23 May 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 40; 2022: 39; 2023: 32; 2024: 27; 2025: 30; 2026: 26.

195 papers.

Sort:

By area:

  1. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  2. 2023Transformers as Algorithms: Generalization and Stability in In-context Learningcited by 222
  3. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusionscited by 214
  4. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 174
  5. 2023The Expressive Power of Low-Rank Adaptationcited by 102
  6. 2021On the Role of Neural Collapse in Transfer Learningcited by 100
  7. 2021On Inductive Biases for Heterogeneous Treatment Effect Estimationcited by 75
  8. 2022FedAvg with Fine Tuning: Local Updates Lead to Representation Learningcited by 56
  9. 2023Revisiting Scalarization in Multi-Task Learning: A Theoretical Perspectivecited by 51
  10. 2021Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptationcited by 50
  11. 2021Near-optimal Representation Learning for Linear Bandits and Linear RLcited by 45
  12. 2021Generalization Bounds For Meta-Learning: An Information-Theoretic Analysiscited by 45
38 more
  1. 2022Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learningcited by 44
  2. 2024Sharing Knowledge in Multi-Task Deep Reinforcement Learningcited by 44
  3. 2024One Policy to Run Them All: an End-to-end Learning Approach to Multi-Embodiment Locomotioncited by 43
  4. 2020Provable Representation Learning for Imitation Learning via Bi-level Optimizationcited by 41
  5. 2021How Fine-Tuning Allows for Effective Meta-Learningcited by 41
  6. 2024An Information-Theoretic Analysis of In-Context Learningcited by 40
  7. 2021Meta-Learning Dynamics Forecasting Using Task Inferencecited by 38
  8. 2022Provable Benefits of Representational Transfer in Reinforcement Learningcited by 38
  9. 2023Deep networks for system identification: a Surveycited by 37
  10. 2021Adversarial Training Helps Transfer Learning via Better Representationscited by 36
  11. 2023The Ideal Continual Learner: An Agent That Never Forgetscited by 35
  12. 2022Adaptive and Robust Multi-Task Learningcited by 34
  13. 2022Combining Observational and Randomized Data for Estimating Heterogeneous Treatment Effectscited by 32
  14. 2021On the Power of Multitask Representation Learning in Linear MDPcited by 30
  15. 2022Provable Benefit of Multitask Representation Learning in Reinforcement Learningcited by 29
  16. 2022Improving Multi-Task Generalization via Regularizing Spurious Correlationcited by 28
  17. 2023Learning from Similar Linear Representations: Adaptivity, Minimaxity, and Robustnesscited by 28
  18. 2024Scaling laws for learning with real and surrogate datacited by 27
  19. 2023A Theory of Multimodal Learningcited by 25
  20. 2021Meta-Adaptive Nonlinear Control: Theory and Algorithmscited by 23
  21. 2022Multi-Task Imitation Learning for Linear Dynamical Systemscited by 23
  22. 2021Weighted Training for Cross-Task Learningcited by 22
  23. 2021How to Train Your MAML to Excel in Few-Shot Classificationcited by 22
  24. 2023On the Provable Advantage of Unsupervised Pretrainingcited by 22
  25. 2021Representation Learning Beyond Linear Prediction Functionscited by 21
  26. 2022Towards Sample-efficient Overparameterized Meta-learningcited by 21
  27. 2022Multi-task Representation Learning with Stochastic Linear Banditscited by 21
  28. 2024Natural Language Processing in the Patent Domain: A Surveycited by 20
  29. 2021Sequoia: A Software Framework to Unify Continual Learning Researchcited by 19
  30. 2023Scalarization for Multi-Task and Multi-Domain Learning at Scalecited by 19
  31. 2024Disentangling and Mitigating the Impact of Task Similarity for Continual Learningcited by 19
  32. 2024Can Large Language Models Generate High-quality Patent Claims?cited by 19
  33. 2021Joint Learning of Linear Time-Invariant Dynamical Systemscited by 16
  34. 2022Nearly Minimax Algorithms for Linear Bandits with Shared Representationcited by 16
  35. 2023Provable Multi-Task Representation Learning by Two-Layer ReLU Neural Networkscited by 16
  36. 2021Learning-to-learn non-convex piecewise-Lipschitz functionscited by 15
  37. 2021A Scaling Law for Synthetic-to-Real Transfer: How Much Is Your Pre-training Effective?cited by 15
  38. 2022Learning Mixtures of Linear Dynamical Systemscited by 15

50 most cited of 195 All 195 (JSON)