# Reinforcement Learning with Convolutional Reservoir Computing

arXiv: https://arxiv.org/abs/1912.04161 · HTML: https://mlsanity.com/paper/1912.04161/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Dec 2019.

## Built on by (most cited first, 11 of 11)

- [A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning](https://mlsanity.com/paper/2307.15092/) (2023)
- [ViR:the Vision Reservoir](https://mlsanity.com/paper/2112.13545/) (2021)
- [Recursive Least Squares Policy Control with Echo State Network](https://mlsanity.com/paper/2201.04781/) (2022)
- [Deep Q-network using reservoir computing with multi-layered readout](https://mlsanity.com/paper/2203.01465/) (2022)
- [A Meta-learning Approach to Reservoir Computing: Time Series Prediction with Limited Data](https://mlsanity.com/paper/2110.03722/) (2021)
- [Evolving Reservoirs for Meta Reinforcement Learning](https://mlsanity.com/paper/2312.06695/) (2023)
- [Multi-agent reinforcement learning using echo-state network and its application to pedestrian dynamics](https://mlsanity.com/paper/2312.11834/) (2023)
- [Chaos-based reinforcement learning with TD3](https://mlsanity.com/paper/2405.09086/) (2024)
- [Efficient training for large-scale optical neural network using an evolutionary strategy and attention pruning](https://mlsanity.com/paper/2505.12906/) (2025)
- [Data-Free Reservoir Features for Efficient Long-Horizon Cold-Start Continual Learning](https://mlsanity.com/paper/2606.27095/) (2026)
- [Emergent Sparsity in Frozen Random CNN Feature Extractors for Deep Reinforcement Learning](https://mlsanity.com/paper/2607.26059/) (2026)

## Abstract (arXiv)

Recently, reinforcement learning models have achieved great success, mastering complex tasks such as Go and other games with higher scores than human players. Many of these models store considerable data on the tasks and achieve high performance by extracting visual and time-series features using convolutional neural networks (CNNs) and recurrent neural networks, respectively. However, these networks have very high computational costs because they need to be trained by repeatedly using the stored data. In this study, we propose a novel practical approach called reinforcement learning with convolutional reservoir computing (RCRC) model. The RCRC model uses a fixed random-weight CNN and a reservoir computing model to extract visual and time-series features. Using these extracted features, it decides actions with an evolution strategy method. Thereby, the RCRC model has several desirable features: (1) there is no need to train the feature extractor, (2) there is no need to store training data, (3) it can take a wide range of actions, and (4) there is only a single task-dependent weight parameter to be trained. Furthermore, we show the RCRC model can solve multiple reinforcement learning tasks with a completely identical feature extractor.