# Continual Learning for non-stationary regression via Memory-Efficient Replay

arXiv: https://arxiv.org/abs/2602.09720 · HTML: https://mlsanity.com/paper/2602.09720/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 39 references that are in the map.
- First posted on arXiv 10 Feb 2026.

## Builds on (most cited first, 24 of 24)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Lifelong Learning with Dynamically Expandable Networks](https://mlsanity.com/paper/1708.01547/) (2017)
- [Continuous Learning in Single-Incremental-Task Scenarios](https://mlsanity.com/paper/1806.08568/) (2018)
- [How catastrophic can catastrophic forgetting be in linear regression?](https://mlsanity.com/paper/2205.09588/) (2022)
- [Maintaining Plasticity in Continual Learning via Regenerative Regularization](https://mlsanity.com/paper/2308.11958/) (2023)
- [SDDGR: Stable Diffusion-based Deep Generative Replay for Class Incremental Object Detection](https://mlsanity.com/paper/2402.17323/) (2024)
- [A Unifying Bayesian View of Continual Learning](https://mlsanity.com/paper/1902.06494/) (2019)
- [Continually Learning Self-Supervised Representations with Projected Functional Regularization](https://mlsanity.com/paper/2112.15022/) (2021)
- [Avalanche: A PyTorch Library for Deep Continual Learning](https://mlsanity.com/paper/2302.01766/) (2023)
- [A Statistical Theory of Regularization-Based Continual Learning](https://mlsanity.com/paper/2406.06213/) (2024)
- [Understanding Forgetting in Continual Learning with Linear Regression](https://mlsanity.com/paper/2405.17583/) (2024)
- [CLeaR: An Adaptive Continual Learning Framework for Regression Tasks](https://mlsanity.com/paper/2101.00926/) (2021)
- [Task-Free Continual Learning via Online Discrepancy Distance Learning](https://mlsanity.com/paper/2210.06579/) (2022)
- [BooVAE: Boosting Approach for Continual Learning of VAE](https://mlsanity.com/paper/1908.11853/) (2019)
- [MalCL: Leveraging GAN-Based Generative Replay to Combat Catastrophic Forgetting in Malware Classification](https://mlsanity.com/paper/2501.01110/) (2025)
- [Continual Learning with Diffusion-based Generative Replay for Industrial Streaming Data](https://mlsanity.com/paper/2406.15766/) (2024)
- [Certified Continual Learning for Neural Network Regression](https://mlsanity.com/paper/2407.06697/) (2024)
- [Overcoming Catastrophic Forgetting in Tabular Data Classification: A Pseudorehearsal-based approach](https://mlsanity.com/paper/2407.09039/) (2024)
- [CFTS-GAN: Continual Few-Shot Teacher Student for Generative Adversarial Networks](https://mlsanity.com/paper/2410.14749/) (2024)
- [Global Convergence of Continual Learning on Non-IID Data](https://mlsanity.com/paper/2503.18511/) (2025)
- [Continual Learning Strategies for 3D Engineering Regression Problems: A Benchmarking Study](https://mlsanity.com/paper/2504.12503/) (2025)
- [Adaptive Explainable Continual Learning Framework for Regression Problems with Focus on Power Forecasts](https://mlsanity.com/paper/2108.10781/) (2021)

## Abstract (arXiv)

Data streams are rarely static in dynamic environments like Industry 4.0. Instead, they constantly change, making traditional offline models outdated unless they can quickly adjust to the new data. This need can be adequately addressed by continual learning (CL), which allows systems to gradually acquire knowledge without incurring the prohibitive costs of retraining them from scratch. Most research on continual learning focuses on classification problems, while very few studies address regression tasks. We propose the first prototype-based generative replay framework designed for online task-free continual regression. Our approach defines an adaptive output-space discretization model, enabling prototype-based generative replay for continual regression without storing raw data. Evidence obtained from several benchmark datasets shows that our framework reduces forgetting and provides more stable performance than other state-of-the-art solutions.