# The difference between memory and prediction in linear recurrent networks

arXiv: https://arxiv.org/abs/1706.09382 · HTML: https://mlsanity.com/paper/1706.09382/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jun 2017.

## Built on by (most cited first, 5 of 5)

- [Exploiting Multiple Timescales in Hierarchical Echo State Networks](https://mlsanity.com/paper/2101.04223/) (2021)
- [Memory of recurrent networks: Do we compute it right?](https://mlsanity.com/paper/2305.01457/) (2023)
- [Memory Capacity of Nonlinear Recurrent Networks: Is it Informative?](https://mlsanity.com/paper/2502.04832/) (2025)
- [Interpretable Design of Reservoir Computing Networks using Realization Theory](https://mlsanity.com/paper/2112.06891/) (2021)
- [Separation capacity of linear reservoirs with random connectivity matrix](https://mlsanity.com/paper/2404.17429/) (2024)

## Abstract (arXiv)

Recurrent networks are trained to memorize their input better, often in the hopes that such training will increase the ability of the network to predict. We show that networks designed to memorize input can be arbitrarily bad at prediction. We also find, for several types of inputs, that one-node networks optimized for prediction are nearly at upper bounds on predictive capacity given by Wiener filters, and are roughly equivalent in performance to randomly generated five-node networks. Our results suggest that maximizing memory capacity leads to very different networks than maximizing predictive capacity, and that optimizing recurrent weights can decrease reservoir size by half an order of magnitude.