mlsanityData to 7 Oct 2026

A comparison of LSTM and GRU networks for learning symbolic sequences

arXiv 2107.02248 ↗First posted 5 Jul 2021cs.LG cs.NEcited by 24

Not on the map. Why?

Abstract (arXiv)

We explore the architecture of recurrent neural networks (RNNs) by studying the complexity of string sequences it is able to memorize. Symbolic sequences of different complexity are generated to simulate RNN training and study parameter configurations with a view to the network's capability of learning and inference. We compare Long Short-Term Memory (LSTM) networks and gated recurrent units (GRUs). We find that an increase in RNN depth does not necessarily result in better memorization capability when the training time is constrained. Our results also indicate that the learning rate and the number of units per layer are among the most important hyper-parameters to be tuned. Generally, GRUs outperform LSTM networks on low-complexity sequences while on high-complexity sequences LSTMs perform better.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 15 of its 36 references that are in the map. First posted on arXiv 5 Jul 2021.

Builds on

15 of 36 references.

References · 15

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  3. 2016An overview of gradient descent optimization algorithmscited by 1,021
  4. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,004
  5. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 744
  6. 2015LSTM: A Search Space Odysseycited by 315
  7. 2019Deep Learning for Symbolic Mathematicscited by 266
  8. 2015Sequence to Sequence -- Video to Textcited by 157
  9. 2014Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN)cited by 94
  10. 2020Principles and Algorithms for Forecasting Groups of Time Series: Locality and Globalitycited by 70
  11. 2017TimeNet: Pre-trained deep recurrent neural network for time series classificationcited by 46
  12. 2020Time Series Forecasting Using LSTM Networks: A Symbolic Approachcited by 25
3 more
  1. 2020The Effectiveness of Discretization in Forecasting: An Empirical Study on Neural Time Series Modelscited by 18
  2. 2014Learning to Discover Efficient Mathematical Identitiescited by 9
  3. 2021Neural Sequence-to-grid Module for Learning Symbolic Rulescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 7; 2024: 4; 2025: 9; 2026: 2.

24 papers.

Sort:

By area:

  1. 2023A Federated Learning-based Industrial Health Prognostics for Heterogeneous Edge Devices using Matched Feature Extractioncited by 12
  2. 2023Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Reviewcited by 9
  3. 2025Advancing Intelligent Sequence Modeling: Evolution, Trade-offs, and Applications of State-Space Architectures from S4 to Mambacited by 9
  4. 2022Cerberus: Exploring Federated Prediction of Security Eventscited by 4
  5. 2025Comprehensive Attribute Encoding and Dynamic LSTM HyperModels for Outcome Oriented Predictive Business Process Monitoringcited by 4
  6. 2024A Comparison of Deep Learning Architectures for Spacecraft Anomaly Detectioncited by 2
  7. 2024ChronoGAN: Supervised and Embedded Generative Adversarial Networks for Time Series Generationcited by 2
  8. 2024Mathematical Formalism for Memory Compression in Selective State Space Modelscited by 2
  9. 2022Modeling and Predicting Transistor Aging under Workload Dependency using Machine Learningcited by 1
  10. 2023RIOT: Recursive Inertial Odometry Transformer for Localisation from Low-Cost IMU Measurementscited by 1
  11. 2023A generalised multi-factor deep learning electricity load forecasting model for wildfire-prone areascited by 1
  12. 2023Mobile Traffic Prediction at the Edge Through Distributed and Deep Transfer Learningcited by 1
12 more
  1. 2025GNN-based Decentralized Perception in Multirobot Systems for Predicting Worker Actionscited by 1
  2. 2025Minion Gated Recurrent Unit for Continual Learningcited by 1
  3. 2025Probing Quantum Spin Systems with Kolmogorov-Arnold Neural Network Quantum Statescited by 1
  4. 2026Robustness of Spatio-temporal Graph Neural Networks for Fault Location in Partially Observable Distribution Gridscited by 1
  5. 2023Using LSTM and GRU With a New Dataset for Named Entity Recognition in the Arabic Languagecited by 0
  6. 2023Deep-seeded Clustering for Emotion Recognition from Wearable Physiological Sensorscited by 0
  7. 2024Enhancing Microgrid Performance Prediction with Attention-based Deep Learning Modelscited by 0
  8. 2025Are ECGs enough? Deep learning classification of pulmonary embolism using electrocardiogramscited by 0
  9. 2025Enhancing Tropical Cyclone Path Forecasting with an Improved Transformer Networkcited by 0
  10. 2025Generalization of Graph Neural Network Models for Distribution Grid Fault Detectioncited by 0
  11. 2025SymSeqBench: a unified framework for the generation and analysis of rule-based symbolic sequences and datasetscited by 0
  12. 2026ROTE: Benchmarking Neural Memorization on Complexity-Controlled Symbolic Sequencescited by 0