# Recurrent Neural Networks (RNNs): A gentle Introduction and Overview

arXiv: https://arxiv.org/abs/1912.05911 · HTML: https://mlsanity.com/paper/1912.05911/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Nov 2019.

## Built on by (most cited first, 30 of 68)

- [Vision-Language Models for Medical Report Generation and Visual Question Answering: A Review](https://mlsanity.com/paper/2403.02469/) (2024)
- [Equivariant quantum circuits for learning on weighted graphs](https://mlsanity.com/paper/2205.06109/) (2022)
- [Foundation Models for Time Series: A Survey](https://mlsanity.com/paper/2504.04011/) (2025)
- [Searching for High-Value Molecules Using Reinforcement Learning and Transformers](https://mlsanity.com/paper/2310.02902/) (2023)
- [Mamba in Vision: A Comprehensive Survey of Techniques and Applications](https://mlsanity.com/paper/2410.03105/) (2024)
- [Large language models for artificial general intelligence (AGI): A survey of foundational principles and approaches](https://mlsanity.com/paper/2501.03151/) (2025)
- [Forecasting steam mass flow in power plants using the parallel hybrid network](https://mlsanity.com/paper/2307.09483/) (2023)
- [Hybrid quantum cycle generative adversarial network for small molecule generation](https://mlsanity.com/paper/2402.00014/) (2023)
- [CuriousLLM: Elevating Multi-Document Question Answering with LLM-Enhanced Knowledge Graph Reasoning](https://mlsanity.com/paper/2404.09077/) (2024)
- [OccamLLM: Fast and Exact Language Model Arithmetic in a Single Step](https://mlsanity.com/paper/2406.06576/) (2024)
- [Evaluation of LLMs for mathematical problem solving](https://mlsanity.com/paper/2506.00309/) (2025)
- [ChunkFormer: Learning Long Time Series with Multi-stage Chunked Transformer](https://mlsanity.com/paper/2112.15087/) (2021)
- [Leveraging the Spatial Hierarchy: Coarse-to-fine Trajectory Generation via Cascaded Hybrid Diffusion](https://mlsanity.com/paper/2507.13366/) (2025)
- [Transfer Learning with Foundational Models for Time Series Forecasting using Low-Rank Adaptations](https://mlsanity.com/paper/2410.11539/) (2024)
- [Optimal Gradient Checkpointing for Sparse and Recurrent Architectures using Off-Chip Memory](https://mlsanity.com/paper/2412.11810/) (2024)
- [IM-BERT: Enhancing Robustness of BERT through the Implicit Euler Method](https://mlsanity.com/paper/2505.06889/) (2025)
- [Spanish Built Factual Freectianary (Spanish-BFF): the first AI-generated free dictionary](https://mlsanity.com/paper/2302.12746/) (2023)
- [Spherical Position Encoding for Transformers](https://mlsanity.com/paper/2310.04454/) (2023)
- [SecureNet: A Comparative Study of DeBERTa and Large Language Models for Phishing Detection](https://mlsanity.com/paper/2406.06663/) (2024)
- [Continual Low-Rank Scaled Dot-product Attention](https://mlsanity.com/paper/2412.03214/) (2024)
- [Can Large Language Models Predict the Outcome of Judicial Decisions?](https://mlsanity.com/paper/2501.09768/) (2025)
- [GAMDTP: Dynamic Trajectory Prediction with Graph Attention Mamba Network](https://mlsanity.com/paper/2504.04862/) (2025)
- [DyTTP: Trajectory Prediction with Normalization-Free Transformers](https://mlsanity.com/paper/2504.05356/) (2025)
- [Hidden State Poisoning Attacks against Mamba-based Language Models](https://mlsanity.com/paper/2601.01972/) (2026)
- [Explainability-aided Domain Generalization for Image Classification](https://mlsanity.com/paper/2104.01742/) (2021)
- [Data Generation for Post-OCR correction of Cyrillic handwriting](https://mlsanity.com/paper/2311.15896/) (2023)
- [A Survey of Classical And Quantum Sequence Models](https://mlsanity.com/paper/2312.10242/) (2023)
- [Distribution-informed and wavelength-flexible data-driven photoacoustic oximetry](https://mlsanity.com/paper/2403.14863/) (2024)
- [Merged ChemProt-DrugProt for Relation Extraction from Biomedical Literature](https://mlsanity.com/paper/2405.18605/) (2024)
- [C-RADAR: A Centralized Deep Learning System for Intrusion Detection in Software Defined Networks](https://mlsanity.com/paper/2408.17356/) (2024)

## Abstract (arXiv)

State-of-the-art solutions in the areas of "Language Modelling & Generating Text", "Speech Recognition", "Generating Image Descriptions" or "Video Tagging" have been using Recurrent Neural Networks as the foundation for their approaches. Understanding the underlying concepts is therefore of tremendous importance if we want to keep up with recent or upcoming publications in those areas. In this work we give a short overview over some of the most important concepts in the realm of Recurrent Neural Networks which enables readers to easily understand the fundamentals such as but not limited to "Backpropagation through Time" or "Long Short-Term Memory Units" as well as some of the more recent advances like the "Attention Mechanism" or "Pointer Networks". We also give recommendations for further reading regarding more complex topics where it is necessary.