# A Comprehensive Overview and Comparative Analysis on Deep Learning Models: CNN, RNN, LSTM, GRU

arXiv: https://arxiv.org/abs/2305.17473 · HTML: https://mlsanity.com/paper/2305.17473/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 116 references that are in the map.
- First posted on arXiv 27 May 2023.

## Builds on (most cited first, 30 of 41)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition](https://mlsanity.com/paper/1406.4729/) (2014)
- [Deep Transfer Learning with Joint Adaptation Networks](https://mlsanity.com/paper/1605.06636/) (2016)
- [Deep Domain Confusion: Maximizing for Domain Invariance](https://mlsanity.com/paper/1412.3474/) (2014)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [A Survey on Deep Transfer Learning](https://mlsanity.com/paper/1808.01974/) (2018)
- [A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects](https://mlsanity.com/paper/2004.02806/) (2020)
- [Adversarial Autoencoders](https://mlsanity.com/paper/1511.05644/) (2015)
- [Recent Advances in Convolutional Neural Networks](https://mlsanity.com/paper/1512.07108/) (2015)
- [Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates](https://mlsanity.com/paper/1610.00633/) (2016)
- [The relativistic discriminator: a key element missing from standard GAN](https://mlsanity.com/paper/1807.00734/) (2018)
- [Beyond Short Snippets: Deep Networks for Video Classification](https://mlsanity.com/paper/1503.08909/) (2015)
- [Coupled Generative Adversarial Networks](https://mlsanity.com/paper/1606.07536/) (2016)
- [Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks](https://mlsanity.com/paper/1506.05751/) (2015)
- [Unrolled Generative Adversarial Networks](https://mlsanity.com/paper/1611.02163/) (2016)
- [Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks](https://mlsanity.com/paper/1604.04382/) (2016)
- [A guide to convolution arithmetic for deep learning](https://mlsanity.com/paper/1603.07285/) (2016)
- [BEGAN: Boundary Equilibrium Generative Adversarial Networks](https://mlsanity.com/paper/1703.10717/) (2017)
- [A Survey on Generative Adversarial Networks: Variants, Applications, and Training](https://mlsanity.com/paper/2006.05132/) (2020)
- [Text-Adaptive Generative Adversarial Networks: Manipulating Images with Natural Language](https://mlsanity.com/paper/1810.11919/) (2018)
- [SteganoGAN: High Capacity Image Steganography with GANs](https://mlsanity.com/paper/1901.03892/) (2019)
- [Adversarial Ranking for Language Generation](https://mlsanity.com/paper/1705.11001/) (2017)
- [Stochastic Pooling for Regularization of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1301.3557/) (2013)
- [Deep Reinforcement Learning in Computer Vision: A Comprehensive Survey](https://mlsanity.com/paper/2108.11510/) (2021)

## Built on by (most cited first, 25 of 25)

- [A Survey on Failure Analysis and Fault Injection in AI Systems](https://mlsanity.com/paper/2407.00125/) (2024)
- [Feature-interactive Siamese graph encoder-based image analysis to predict STAS from histopathology images in lung cancer](https://mlsanity.com/paper/2411.15274/) (2024)
- [Time Series Stock Price Forecasting Based on Genetic Algorithm (GA)-Long Short-Term Memory Network (LSTM) Optimization](https://mlsanity.com/paper/2405.03151/) (2024)
- [Recent Advances in Federated Learning Driven Large Language Models: A Survey on Architecture, Performance, and Security](https://mlsanity.com/paper/2406.09831/) (2024)
- [Explainable Artificial Intelligence Techniques for Irregular Temporal Classification of Multidrug Resistance Acquisition in Intensive Care Unit Patients](https://mlsanity.com/paper/2407.17165/) (2024)
- [Optimizing Vision Transformers with Data-Free Knowledge Transfer](https://mlsanity.com/paper/2408.05952/) (2024)
- [A Novel Approach to Malicious Code Detection Using CNN-BiLSTM and Feature Fusion](https://mlsanity.com/paper/2410.09401/) (2024)
- [Machine learning approach to brain tumor detection and classification](https://mlsanity.com/paper/2410.12692/) (2024)
- [Deep Learning Based XIoT Malware Analysis: A Comprehensive Survey, Taxonomy, and Research Challenges](https://mlsanity.com/paper/2410.13894/) (2024)
- [A Temporal Convolutional Network-based Approach for Network Intrusion Detection](https://mlsanity.com/paper/2412.17452/) (2024)
- [SignBart -- New approach with the skeleton sequence for Isolated Sign language Recognition](https://mlsanity.com/paper/2506.21592/) (2025)
- [Data-Driven Dimensional Synthesis of Diverse Planar Four-bar Function Generation Mechanisms via Direct Parameterization](https://mlsanity.com/paper/2507.08269/) (2025)
- [TinyD'ej`aVu: Smaller RAM and Faster Inference with Neural Networks on MCUs for Sensor Data Streams](https://mlsanity.com/paper/2512.09786/) (2025)
- [Accelerated Digital Twin Learning for Edge AI: A Comparison of FPGA and Mobile GPU](https://mlsanity.com/paper/2512.17941/) (2025)
- [AI-Powered Energy Algorithmic Trading: Integrating Hidden Markov Models with Neural Networks](https://mlsanity.com/paper/2407.19858/) (2024)
- [Smart Data-Driven GRU Predictor for SnO2 Thin films Characteristics](https://mlsanity.com/paper/2409.11782/) (2024)
- [Evaluating Neural Networks for Early Maritime Threat Detection](https://mlsanity.com/paper/2410.20054/) (2024)
- [A Multi-Agent Reinforcement Learning Approach for Cooperative Air-Ground-Human Crowdsensing in Emergency Rescue](https://mlsanity.com/paper/2505.06997/) (2025)
- [A Laplace diffusion-based transformer model for heart rate forecasting within daily activity context](https://mlsanity.com/paper/2508.16655/) (2025)
- [Skeleton-based sign language recognition using a dual-stream spatio-temporal dynamic graph convolutional network](https://mlsanity.com/paper/2509.08661/) (2025)
- [Hydrogen production from blended waste biomass: pyrolysis, thermodynamic-kinetic analysis and AI-based modelling](https://mlsanity.com/paper/2510.15960/) (2025)
- [UniCoMTE: A Universal Counterfactual Framework for Explaining Time-Series Classifiers on ECG Data](https://mlsanity.com/paper/2512.17100/) (2025)
- [Leveraging Physiological Signals to Predict Exam Outcomes with Machine Learning](https://mlsanity.com/paper/2606.14960/) (2026)
- [Automatic Field-of-View Adjustment for a View-Expansive Microscope via LSTM-Based Gaze and Pipette Motion Interpretation](https://mlsanity.com/paper/2608.10401/) (2026)
- [FLY-EVAL++: An Evidence-Driven Evaluation Protocol for Safety-Constrained Flight Prediction with Large Language Models](https://mlsanity.com/paper/2609.04021/) (2026)

## Abstract (arXiv)

Deep learning (DL) has emerged as a powerful subset of machine learning (ML) and artificial intelligence (AI), outperforming traditional ML methods, especially in handling unstructured and large datasets. Its impact spans across various domains, including speech recognition, healthcare, autonomous vehicles, cybersecurity, predictive analytics, and more. However, the complexity and dynamic nature of real-world problems present challenges in designing effective deep learning models. Consequently, several deep learning models have been developed to address different problems and applications. In this article, we conduct a comprehensive survey of various deep learning models, including Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Temporal Convolutional Networks (TCN), Transformer, Kolmogorov-Arnold networks (KAN), Generative Models, Deep Reinforcement Learning (DRL), and Deep Transfer Learning. We examine the structure, applications, benefits, and limitations of each model. Furthermore, we perform an analysis using three publicly available datasets: IMDB, ARAS, and Fruit-360. We compared the performance of six renowned deep learning models: CNN, RNN, Long Short-Term Memory (LSTM), Bidirectional LSTM, Gated Recurrent Unit (GRU), and Bidirectional GRU alongside two newer models, TCN and Transformer, using the IMDB and ARAS datasets. Additionally, we evaluated the performance of eight CNN-based models, including VGG (Visual Geometry Group), Inception, ResNet (Residual Network), InceptionResNet, Xception (Extreme Inception), MobileNet, DenseNet (Dense Convolutional Network), and NASNet (Neural Architecture Search Network), for image classification tasks using the Fruit-360 dataset.