# Regret Bounds for Lifelong Learning

arXiv: https://arxiv.org/abs/1610.08628 · HTML: https://mlsanity.com/paper/1610.08628/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Oct 2016.

## Built on by (most cited first, 19 of 19)

- [How Fine-Tuning Allows for Effective Meta-Learning](https://mlsanity.com/paper/2105.02221/) (2021)
- [Generalization Bounds for Meta-Learning via PAC-Bayes and Uniform Stability](https://mlsanity.com/paper/2102.06589/) (2021)
- [An Online Learning Approach to Interpolation and Extrapolation in Domain Generalization](https://mlsanity.com/paper/2102.13128/) (2021)
- [On the Power of Multitask Representation Learning in Linear MDP](https://mlsanity.com/paper/2106.08053/) (2021)
- [Joint Learning of Linear Time-Invariant Dynamical Systems](https://mlsanity.com/paper/2112.10955/) (2021)
- [Nearly Minimax Algorithms for Linear Bandits with Shared Representation](https://mlsanity.com/paper/2203.15664/) (2022)
- [Learning-to-learn non-convex piecewise-Lipschitz functions](https://mlsanity.com/paper/2108.08770/) (2021)
- [Provable Continual Learning via Sketched Jacobian Approximations](https://mlsanity.com/paper/2112.05095/) (2021)
- [A Representation Learning Perspective on the Importance of Train-Validation Splitting in Meta-Learning](https://mlsanity.com/paper/2106.15615/) (2021)
- [A CMDP-within-online framework for Meta-Safe Reinforcement Learning](https://mlsanity.com/paper/2405.16601/) (2024)
- [Bayes meets Bernstein at the Meta Level: an Analysis of Fast Rates in Meta-Learning with PAC-Bayes](https://mlsanity.com/paper/2302.11709/) (2023)
- [Meta-strategy for Learning Tuning Parameters with Guarantees](https://mlsanity.com/paper/2102.02504/) (2021)
- [Multitask Online Mirror Descent](https://mlsanity.com/paper/2106.02393/) (2021)
- [On the Sample Complexity of Representation Learning in Multi-task Bandits with Global and Local structure](https://mlsanity.com/paper/2211.15129/) (2022)
- [Multi-Environment Meta-Learning in Stochastic Linear Bandits](https://mlsanity.com/paper/2205.06326/) (2022)
- [AdaTask: Adaptive Multitask Online Learning](https://mlsanity.com/paper/2205.15802/) (2022)
- [Prototypical quadruplet for few-shot class incremental learning](https://mlsanity.com/paper/2211.02947/) (2022)
- [On regret bounds for continual single-index learning](https://mlsanity.com/paper/2102.12961/) (2021)
- [Bridging Lifelong and Multi-Task Representation Learning via Algorithm and Complexity Measure](https://mlsanity.com/paper/2511.01847/) (2025)

## Abstract (arXiv)

We consider the problem of transfer learning in an online setting. Different tasks are presented sequentially and processed by a within-task algorithm. We propose a lifelong learning strategy which refines the underlying data representation used by the within-task algorithm, thereby transferring information from one task to the next. We show that when the within-task algorithm comes with some regret bound, our strategy inherits this good property. Our bounds are in expectation for a general loss function, and uniform for a convex loss. We discuss applications to dictionary learning and finite set of predictors. In the latter case, we improve previous O(1/√m) bounds to O(1/m) where m is the per task sample size.