mlsanityData to 14 Oct 2026

Bridging Lifelong and Multi-Task Representation Learning via Algorithm and Complexity Measure

arXiv 2511.01847 ↗First posted 3 Nov 2025cs.LG stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

In lifelong learning, a learner faces a sequence of tasks with shared structure and aims to identify and leverage it to accelerate learning. We study the setting where such structure is captured by a common representation of data. Unlike multi-task learning or learning-to-learn, where tasks are available upfront to learn the representation, lifelong learning requires the learner to make use of its existing knowledge while continually gathering partial information in an online fashion. In this paper, we consider a generalized framework of lifelong representation learning. We propose a simple algorithm that uses multi-task empirical risk minimization as a subroutine and establish a sample complexity bound based on a new notion we introduce--the task-eluder dimension. Our result applies to a wide range of learning problems involving general function classes. As concrete examples, we instantiate our result on classification and regression tasks under noise.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 34 of its 60 references that are in the map. First posted on arXiv 3 Nov 2025.

Builds on

34 of 60 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  4. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  5. 2017A Survey on Multi-Task Learningcited by 840
  6. 2013DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognitioncited by 424
  7. 2018Selective Experience Replay for Lifelong Learningcited by 241
  8. 2020Few-Shot Learning via Learning the Representation, Provablycited by 214
  9. 2015The Benefit of Multitask Representation Learningcited by 195
  10. 2019Episodic Memory in Lifelong Language Learningcited by 188
  11. 2020On the Theory of Transfer Learning: The Importance of Task Diversitycited by 186
  12. 2020Provable Meta-Learning of Linear Representationscited by 161
22 more
  1. 2019Online Meta-Learningcited by 137
  2. 2014Model-based Reinforcement Learning and the Eluder Dimensioncited by 116
  3. 2018Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Controlcited by 113
  4. 2020Instance-Dependent Complexity of Contextual Bandits and Reinforcement Learning: A Disagreement-Based Perspectivecited by 78
  5. 2013A PAC-Bayesian bound for Lifelong Learningcited by 69
  6. 2020Impact of Representation Learning in Linear Banditscited by 52
  7. 2021Near-optimal Representation Learning for Linear Bandits and Linear RLcited by 45
  8. 2014Efficient Representations for Life-Long Learning and Autoencodingcited by 23
  9. 2021Representation Learning Beyond Linear Prediction Functionscited by 21
  10. 2016Regret Bounds for Lifelong Learningcited by 19
  11. 2021Understanding the Eluder Dimensioncited by 19
  12. 2022Nearly Minimax Algorithms for Linear Bandits with Shared Representationcited by 16
  13. 2021Provable Lifelong Learning of Representationscited by 15
  14. 2022Active Multi-Task Representation Learningcited by 14
  15. 2022Provable and Efficient Continual Representation Learningcited by 14
  16. 2014A chain rule for the expected suprema of Gaussian processescited by 13
  17. 2022Non-Stationary Representation Learning in Sequential Linear Banditscited by 13
  18. 2022An Introduction to Lifelong Supervised Learningcited by 12
  19. 2020Noise in Classificationcited by 11
  20. 2023Metalearning with Very Few Samples Per Taskcited by 11
  21. 2018Estimating Learnability in the Sublinear Data Regimecited by 10
  22. 2016Multi-task and Lifelong Learning of Kernelscited by 7

Built on by

Not cited yet.