# Investigating Recurrent Neural Network Memory Structures using Neuro-Evolution

arXiv: https://arxiv.org/abs/1902.02390 · HTML: https://mlsanity.com/paper/1902.02390/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Feb 2019.

## Built on by (most cited first, 9 of 9)

- [Fitness Landscape Footprint: A Framework to Compare Neural Architecture Search Problems](https://mlsanity.com/paper/2111.01584/) (2021)
- [ONE-NAS: An Online NeuroEvolution based Neural Architecture Search for Time Series Forecasting](https://mlsanity.com/paper/2202.13471/) (2022)
- [A Data-driven Approach to Neural Architecture Search Initialization](https://mlsanity.com/paper/2111.03524/) (2021)
- [Online Evolutionary Neural Architecture Search for Multivariate Non-Stationary Time Series Forecasting](https://mlsanity.com/paper/2302.10347/) (2023)
- [Neuroevolution Neural Architecture Search for Evolving RNNs in Stock Return Prediction and Portfolio Trading](https://mlsanity.com/paper/2410.17212/) (2024)
- [Investigating Quantum Circuit Designs Using Neuro-Evolution](https://mlsanity.com/paper/2602.03840/) (2026)
- [Reliable and Fast Recurrent Neural Network Architecture Optimization](https://mlsanity.com/paper/2106.15295/) (2021)
- [Addressing Tactic Volatility in Self-Adaptive Systems Using Evolved Recurrent Neural Networks and Uncertainty Reduction Tactics](https://mlsanity.com/paper/2204.10308/) (2022)
- [EVOTS: Evolutionary Transformer Search for Time Series Forecasting](https://mlsanity.com/paper/2607.00154/) (2026)

## Abstract (arXiv)

This paper presents a new algorithm, Evolutionary eXploration of Augmenting Memory Models (EXAMM), which is capable of evolving recurrent neural networks (RNNs) using a wide variety of memory structures, such as Delta-RNN, GRU, LSTM, MGU and UGRNN cells. EXAMM evolved RNNs to perform prediction of large-scale, real world time series data from the aviation and power industries. These data sets consist of very long time series (thousands of readings), each with a large number of potentially correlated and dependent parameters. Four different parameters were selected for prediction and EXAMM runs were performed using each memory cell type alone, each cell type with feed forward nodes, and with all possible memory cell types. Evolved RNN performance was measured using repeated k-fold cross validation, resulting in 1210 EXAMM runs which evolved 2,420,000 RNNs in 12,100 CPU hours on a high performance computing cluster. Generalization of the evolved RNNs was examined statistically, providing interesting findings that can help refine the RNN memory cell design as well as inform future neuro-evolution algorithms development.