# Continual Learning with Gated Incremental Memories for sequential data processing

arXiv: https://arxiv.org/abs/2004.04077 · HTML: https://mlsanity.com/paper/2004.04077/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Apr 2020.

## Built on by (most cited first, 12 of 12)

- [GCR: Gradient Coreset Based Replay Buffer Selection For Continual Learning](https://mlsanity.com/paper/2111.11210/) (2021)
- [Distilled Replay: Overcoming Forgetting through Synthetic Samples](https://mlsanity.com/paper/2103.15851/) (2021)
- [Continual Learning for Recurrent Neural Networks: an Empirical Evaluation](https://mlsanity.com/paper/2103.07492/) (2021)
- [Catastrophic Forgetting in Deep Graph Networks: an Introductory Benchmark for Graph Classification](https://mlsanity.com/paper/2103.11750/) (2021)
- [Class-incremental Learning for Time Series: Benchmark and Evaluation](https://mlsanity.com/paper/2402.12035/) (2024)
- [Continual Learning with Echo State Networks](https://mlsanity.com/paper/2105.07674/) (2021)
- [Efficient and robust multi-task learning in the brain with modular latent primitives](https://mlsanity.com/paper/2105.14108/) (2021)
- [TEACHING -- Trustworthy autonomous cyber-physical applications through human-centred intelligence](https://mlsanity.com/paper/2107.06543/) (2021)
- [cPNN: Continuous Progressive Neural Networks for Evolving Streaming Time Series](https://mlsanity.com/paper/2603.03040/) (2026)
- [Towards lifelong learning of Recurrent Neural Networks for control design](https://mlsanity.com/paper/2208.03980/) (2022)
- [MAcPNN: Mutual Assisted Learning on Data Streams with Temporal Dependence](https://mlsanity.com/paper/2603.08972/) (2026)
- [Towards Continuous Power Forecasting: Practical Continual Learning for Real-World Energy Systems in Nonstationary Time Series](https://mlsanity.com/paper/2606.24955/) (2026)

## Abstract (arXiv)

The ability to learn in dynamic, nonstationary environments without forgetting previous knowledge, also known as Continual Learning (CL), is a key enabler for scalable and trustworthy deployments of adaptive solutions. While the importance of continual learning is largely acknowledged in machine vision and reinforcement learning problems, this is mostly under-documented for sequence processing tasks. This work proposes a Recurrent Neural Network (RNN) model for CL that is able to deal with concept drift in input distribution without forgetting previously acquired knowledge. We also implement and test a popular CL approach, Elastic Weight Consolidation (EWC), on top of two different types of RNNs. Finally, we compare the performances of our enhanced architecture against EWC and RNNs on a set of standard CL benchmarks, adapted to the sequential data processing scenario. Results show the superior performance of our architecture and highlight the need for special solutions designed to address CL in RNNs.