mlsanityData to 7 Oct 2026

A PAC-Bayesian bound for Lifelong Learning

arXiv 1311.2838 ↗First posted 12 Nov 2013stat.ML cs.LGcited by 69

Not on the map. Why?

Abstract (arXiv)

Transfer learning has received a lot of attention in the machine learning community over the last years, and several effective algorithms have been developed. However, relatively little is known about their theoretical properties, especially in the setting of lifelong learning, where the goal is to transfer information to tasks for which no data have been observed so far. In this work we study lifelong learning from a theoretical perspective. Our main result is a PAC-Bayesian generalization bound that offers a unified view on existing paradigms for transfer learning, such as the transfer of parameters or the transfer of low-dimensional representations. We also use the bound to derive two principled lifelong learning algorithms, and we show that these yield results comparable with existing methods.

Key facts. Cited by 69 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Nov 2013.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 23; 2022: 14; 2023: 13; 2024: 8; 2025: 7; 2026: 4.

69 papers.

Sort:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2021User-friendly introduction to PAC-Bayes boundscited by 234
  3. 2021On the Role of Neural Collapse in Transfer Learningcited by 100
  4. 2022An Information-Theoretic Approach to Transferability in Task Transfer Learningcited by 79
  5. 2022A Theoretical Study on Solving Continual Learningcited by 78
  6. 2023A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learningcited by 66
  7. 2023Continual Learning: Applications and the Road Forwardcited by 59
  8. 2021Lifelong Person Re-Identification via Adaptive Knowledge Accumulationcited by 57
  9. 2021Layerwise Optimization by Gradient Decomposition for Continual Learningcited by 51
  10. 2021AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learningcited by 46
  11. 2021Generalization Bounds For Meta-Learning: An Information-Theoretic Analysiscited by 45
  12. 2021How Fine-Tuning Allows for Effective Meta-Learningcited by 41
38 more
  1. 2021From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligencecited by 35
  2. 2022On Learning Fairness and Accuracy on Multiple Subgroupscited by 31
  3. 2021Generalization Bounds for Meta-Learning via PAC-Bayes and Uniform Stabilitycited by 29
  4. 2023Learnability and Algorithm for Continual Learningcited by 28
  5. 2021Learngene: From Open-World to Your Learning Taskcited by 27
  6. 2023A Theory of Multimodal Learningcited by 25
  7. 2021Meta-Learning Reliable Priors in the Function Spacecited by 23
  8. 2023Open-World Continual Learning: Unifying Novelty Detection and Continual Learningcited by 22
  9. 2021Bridging the Gap Between Practice and PAC-Bayes Theory in Few-Shot Meta-Learningcited by 21
  10. 2023PeFLL: Personalized Federated Learning by Learning to Learncited by 21
  11. 2021Metalearning Linear Bandits by Prior Updatecited by 15
  12. 2021Learning Curves for Continual Learning in Neural Networks: Self-Knowledge Transfer and Forgettingcited by 14
  13. 2022Meta-Learning Priors for Safe Bayesian Optimizationcited by 14
  14. 2021Conditional Meta-Learning of Linear Representationscited by 12
  15. 2021A Representation Learning Perspective on the Importance of Train-Validation Splitting in Meta-Learningcited by 12
  16. 2021PAC-Bayes Bounds for Meta-learning with Data-Dependent Priorcited by 11
  17. 2022Evaluated CMI Bounds for Meta Learning: Tightness and Expressivenesscited by 11
  18. 2022Scalable PAC-Bayesian Meta-Learning via the PAC-Optimal Hyper-Posterior: From Theory to Practicecited by 11
  19. 2023Metalearning with Very Few Samples Per Taskcited by 11
  20. 2021An Information-Theoretic Analysis of the Impact of Task Similarity on Meta-Learningcited by 10
  21. 2021Information-Theoretic Analysis of Epistemic Uncertainty in Bayesian Meta-learningcited by 10
  22. 2024More Flexible PAC-Bayesian Meta-Learning by Learning Learning Algorithmscited by 10
  23. 2021Learning an Explicit Hyperparameter Prediction Function Conditioned on Taskscited by 9
  24. 2023Limits of Model Selection under Transfer Learningcited by 9
  25. 2023Bayes meets Bernstein at the Meta Level: an Analysis of Fast Rates in Meta-Learning with PAC-Bayescited by 8
  26. 2022PAC-Bayes Bounds for Bandit Problems: A Survey and Experimental Comparisoncited by 7
  27. 2022Generalization Bounds for Few-Shot Transfer Learning with Pretrained Classifierscited by 7
  28. 2023Best-Effort Adaptationcited by 7
  29. 2021Meta-strategy for Learning Tuning Parameters with Guaranteescited by 6
  30. 2023Data-Efficient Task Generalization via Probabilistic Model-based Meta Reinforcement Learningcited by 6
  31. 2026Self-Evolving Agents with Anytime-Valid Certificatescited by 6
  32. 2022Generalization Properties of Retrieval-based Modelscited by 5
  33. 2022MARS: Meta-Learning as Score Matching in the Function Spacecited by 5
  34. 2024Personalized Federated Learning of Probabilistic Models: A PAC-Bayesian Approachcited by 5
  35. 2025From Low Intrinsic Dimensionality to Non-Vacuous Generalization Bounds in Deep Multi-Task Learningcited by 5
  36. 2025Measuring Representational Shifts in Continual Learning: A Linear Transformation Perspectivecited by 5
  37. 2022PAC-Bayesian Lifelong Learning For Multi-Armed Banditscited by 4
  38. 2022A General framework for PAC-Bayes Bounds for Meta-Learningcited by 3

50 most cited of 69 All 69 (JSON)