# Bilevel Continual Learning

arXiv: https://arxiv.org/abs/2007.15553 · HTML: https://mlsanity.com/paper/2007.15553/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jul 2020.

## Built on by (most cited first, 11 of 11)

- [Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond](https://mlsanity.com/paper/2101.11517/) (2021)
- [DualNet: Continual Learning, Fast and Slow](https://mlsanity.com/paper/2110.00175/) (2021)
- [Inexact bilevel stochastic gradient methods for constrained and unconstrained lower-level problems](https://mlsanity.com/paper/2110.00604/) (2021)
- [Online Continual Learning via the Meta-learning Update with Multi-scale Knowledge Distillation and Data Augmentation](https://mlsanity.com/paper/2209.06107/) (2022)
- [Fully Zeroth-Order Bilevel Programming via Gaussian Smoothing](https://mlsanity.com/paper/2404.00158/) (2024)
- [A Unified Gradient-based Framework for Task-agnostic Continual Learning-Unlearning](https://mlsanity.com/paper/2505.15178/) (2025)
- [Selecting Related Knowledge via Efficient Channel Attention for Online Continual Learning](https://mlsanity.com/paper/2209.04212/) (2022)
- [Bilevel Online Deep Learning in Non-stationary Environment](https://mlsanity.com/paper/2201.10236/) (2022)
- [Online Continual Learning via the Knowledge Invariant and Spread-out Properties](https://mlsanity.com/paper/2302.00858/) (2023)
- [Continual Learning, Fast and Slow](https://mlsanity.com/paper/2209.02370/) (2022)
- [Overcoming Catastrophic Forgetting: Continual Learning for Radar Waveform Classification](https://mlsanity.com/paper/2610.18404/) (2026)

## Abstract (arXiv)

Continual learning aims to learn continuously from a stream of tasks and data in an online-learning fashion, being capable of exploiting what was learned previously to improve current and future tasks while still being able to perform well on the previous tasks. One common limitation of many existing continual learning methods is that they often train a model directly on all available training data without validation due to the nature of continual learning, thus suffering poor generalization at test time. In this work, we present a novel framework of continual learning named "Bilevel Continual Learning" (BCL) by unifying a it bilevel optimization objective and a it dual memory management strategy comprising both episodic memory and generalization memory to achieve effective knowledge transfer to future tasks and alleviate catastrophic forgetting on old tasks simultaneously. Our extensive experiments on continual learning benchmarks demonstrate the efficacy of the proposed BCL compared to many state-of-the-art methods. Our implementation is available at https://github.com/phquang/bilevel-continual-learning.