# Continual Learning with Echo State Networks

arXiv: https://arxiv.org/abs/2105.07674 · HTML: https://mlsanity.com/paper/2105.07674/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 14 references that are in the map.
- First posted on arXiv 17 May 2021.

## Builds on (most cited first, 8 of 8)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Avalanche: an End-to-End Library for Continual Learning](https://mlsanity.com/paper/2104.00405/) (2021)
- [Continual Learning for Robotics: Definition, Framework, Learning Strategies, Opportunities and Challenges](https://mlsanity.com/paper/1907.00182/) (2019)
- [Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis](https://mlsanity.com/paper/1909.01520/) (2019)
- [Continual Learning for Recurrent Neural Networks: an Empirical Evaluation](https://mlsanity.com/paper/2103.07492/) (2021)
- [Spiking Neural Predictive Coding for Continual Learning from Data Streams](https://mlsanity.com/paper/1908.08655/) (2019)
- [Continual Learning with Gated Incremental Memories for sequential data processing](https://mlsanity.com/paper/2004.04077/) (2020)

## Built on by (most cited first, 8 of 8)

- [Spiking Neural Networks and Bio-Inspired Supervised Deep Learning: A Survey](https://mlsanity.com/paper/2307.16235/) (2023)
- [Continual Learning of Dynamical Systems with Competitive Federated Reservoir Computing](https://mlsanity.com/paper/2206.13336/) (2022)
- [TEACHING -- Trustworthy autonomous cyber-physical applications through human-centred intelligence](https://mlsanity.com/paper/2107.06543/) (2021)
- [Continual Learning for Human State Monitoring](https://mlsanity.com/paper/2207.00010/) (2022)
- [A Protocol for Continual Explanation of SHAP](https://mlsanity.com/paper/2306.07218/) (2023)
- [From Neurons to Computation: Biological Reservoir Computing for Pattern Recognition](https://mlsanity.com/paper/2505.03510/) (2025)
- [From Neural Activity to Computation: Biological Reservoirs for Pattern Recognition in Digit Classification](https://mlsanity.com/paper/2510.05637/) (2025)
- [Neuro-Inspired Visual Pattern Recognition via Biological Reservoir Computing](https://mlsanity.com/paper/2602.05737/) (2026)

## Abstract (arXiv)

Continual Learning (CL) refers to a learning setup where data is non stationary and the model has to learn without forgetting existing knowledge. The study of CL for sequential patterns revolves around trained recurrent networks. In this work, instead, we introduce CL in the context of Echo State Networks (ESNs), where the recurrent component is kept fixed. We provide the first evaluation of catastrophic forgetting in ESNs and we highlight the benefits in using CL strategies which are not applicable to trained recurrent models. Our results confirm the ESN as a promising model for CL and open to its use in streaming scenarios.