# CLeaR: An Adaptive Continual Learning Framework for Regression Tasks

arXiv: https://arxiv.org/abs/2101.00926 · HTML: https://mlsanity.com/paper/2101.00926/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 33 references that are in the map.
- First posted on arXiv 4 Jan 2021.

## Builds on (most cited first, 16 of 16)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence](https://mlsanity.com/paper/1801.10112/) (2018)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [Progress & Compress: A scalable framework for continual learning](https://mlsanity.com/paper/1805.06370/) (2018)
- [Variational Continual Learning](https://mlsanity.com/paper/1710.10628/) (2017)
- [CORe50: a New Dataset and Benchmark for Continuous Object Recognition](https://mlsanity.com/paper/1705.03550/) (2017)
- [Towards Robust Evaluations of Continual Learning](https://mlsanity.com/paper/1805.09733/) (2018)
- [Continuous Learning in Single-Incremental-Task Scenarios](https://mlsanity.com/paper/1806.08568/) (2018)
- [Generative replay with feedback connections as a general strategy for continual learning](https://mlsanity.com/paper/1809.10635/) (2018)
- [Less-forgetting Learning in Deep Neural Networks](https://mlsanity.com/paper/1607.00122/) (2016)
- [On Quadratic Penalties in Elastic Weight Consolidation](https://mlsanity.com/paper/1712.03847/) (2017)
- [A Unifying Bayesian View of Continual Learning](https://mlsanity.com/paper/1902.06494/) (2019)

## Built on by (most cited first, 22 of 22)

- [Learning Fast and Slow for Online Time Series Forecasting](https://mlsanity.com/paper/2202.11672/) (2022)
- [Elephant Neural Networks: Born to Be a Continual Learner](https://mlsanity.com/paper/2310.01365/) (2023)
- [Forget Less, Count Better: A Domain-Incremental Self-Distillation Learning Benchmark for Lifelong Crowd Counting](https://mlsanity.com/paper/2205.03307/) (2022)
- [BeGin: Extensive Benchmark Scenarios and An Easy-to-use Framework for Graph Continual Learning](https://mlsanity.com/paper/2211.14568/) (2022)
- [Structural Knowledge Informed Continual Multivariate Time Series Forecasting](https://mlsanity.com/paper/2402.12722/) (2024)
- [Continual Learning in Predictive Autoscaling](https://mlsanity.com/paper/2307.15941/) (2023)
- [Continual Learning Strategies for 3D Engineering Regression Problems: A Benchmarking Study](https://mlsanity.com/paper/2504.12503/) (2025)
- [SLE-FNO: Single-Layer Extensions for Task-Agnostic Continual Learning in Fourier Neural Operators](https://mlsanity.com/paper/2603.20410/) (2026)
- [Adaptive Explainable Continual Learning Framework for Regression Problems with Focus on Power Forecasts](https://mlsanity.com/paper/2108.10781/) (2021)
- [Mixture of basis for interpretable continual learning with distribution shifts](https://mlsanity.com/paper/2201.01853/) (2022)
- [Real World Time Series Benchmark Datasets with Distribution Shifts: Global Crude Oil Price and Volatility](https://mlsanity.com/paper/2308.10846/) (2023)
- [A Self-Commissioning Edge Computing Method for Data-Driven Anomaly Detection in Power Electronic Systems](https://mlsanity.com/paper/2312.02661/) (2023)
- [ACIL: Active Class Incremental Learning for Image Classification](https://mlsanity.com/paper/2602.04252/) (2026)
- [Continual Learning of Feedback-based Molecular Communication](https://mlsanity.com/paper/2605.01020/) (2026)
- [Artificial intelligence for online characterization of ultrashort X-ray free-electron laser pulses](https://mlsanity.com/paper/2108.13979/) (2021)
- [Design of Explainability Module with Experts in the Loop for Visualization and Dynamic Adjustment of Continual Learning](https://mlsanity.com/paper/2202.06781/) (2022)
- [ODEStream: A Buffer-Free Online Learning Framework with ODE-based Adaptor for Streaming Time Series Forecasting](https://mlsanity.com/paper/2411.07413/) (2024)
- [Continual Learning for non-stationary regression via Memory-Efficient Replay](https://mlsanity.com/paper/2602.09720/) (2026)
- [Locally Linear Continual Learning for Time Series based on VC-Theoretical Generalization Bounds](https://mlsanity.com/paper/2603.13674/) (2026)
- [Towards Continuous Power Forecasting: Practical Continual Learning for Real-World Energy Systems in Nonstationary Time Series](https://mlsanity.com/paper/2606.24955/) (2026)
- [Attention-based Experience Replay Framework for Continual Learning of Agnostic Time Series Forecasting Models](https://mlsanity.com/paper/2607.20493/) (2026)
- [Continual Learning for Reliable In-Sequence Molecular Communication](https://mlsanity.com/paper/2610.20426/) (2026)

## Abstract (arXiv)

Catastrophic forgetting means that a trained neural network model gradually forgets the previously learned tasks when being retrained on new tasks. Overcoming the forgetting problem is a major problem in machine learning. Numerous continual learning algorithms are very successful in incremental learning of classification tasks, where new samples with their labels appear frequently. However, there is currently no research that addresses the catastrophic forgetting problem in regression tasks as far as we know. This problem has emerged as one of the primary constraints in some applications, such as renewable energy forecasts. This article clarifies problem-related definitions and proposes a new methodological framework that can forecast targets and update itself by means of continual learning. The framework consists of forecasting neural networks and buffers, which store newly collected data from a non-stationary data stream in an application. The changed probability distribution of the data stream, which the framework has identified, will be learned sequentially. The framework is called CLeaR (Continual Learning for Regression Tasks), where components can be flexibly customized for a specific application scenario. We design two sets of experiments to evaluate the CLeaR framework concerning fitting error (training), prediction error (test), and forgetting ratio. The first one is based on an artificial time series to explore how hyperparameters affect the CLeaR framework. The second one is designed with data collected from European wind farms to evaluate the CLeaR framework's performance in a real-world application. The experimental results demonstrate that the CLeaR framework can continually acquire knowledge in the data stream and improve the prediction accuracy. The article concludes with further research issues arising from requirements to extend the framework.