# The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches

arXiv: https://arxiv.org/abs/1803.01164 · HTML: https://mlsanity.com/paper/1803.01164/

## Key facts

- Cited by 75 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Mar 2018.

## Built on by (most cited first, 30 of 75)

- [AI and the Everything in the Whole Wide World Benchmark](https://mlsanity.com/paper/2111.15366/) (2021)
- [RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation based on Visual Foundation Model](https://mlsanity.com/paper/2306.16269/) (2023)
- [Compute Trends Across Three Eras of Machine Learning](https://mlsanity.com/paper/2202.05924/) (2022)
- [FedCLIP: Fast Generalization and Personalization for CLIP in Federated Learning](https://mlsanity.com/paper/2302.13485/) (2023)
- [Multiple Instance Captioning: Learning Representations from Histopathology Textbooks and Articles](https://mlsanity.com/paper/2103.05121/) (2021)
- [Introduction to Latent Variable Energy-Based Models: A Path Towards Autonomous Machine Intelligence](https://mlsanity.com/paper/2306.02572/) (2023)
- [Artificial Intelligence in Drug Discovery: Applications and Techniques](https://mlsanity.com/paper/2106.05386/) (2021)
- [CausalRivers -- Scaling up benchmarking of causal discovery for real-world time-series](https://mlsanity.com/paper/2503.17452/) (2025)
- [Server-Side Local Gradient Averaging and Learning Rate Acceleration for Scalable Split Learning](https://mlsanity.com/paper/2112.05929/) (2021)
- [A review of machine learning in processing remote sensing data for mineral exploration](https://mlsanity.com/paper/2103.07678/) (2021)
- [SymbolicAI: A framework for logic-based approaches combining generative models and solvers](https://mlsanity.com/paper/2402.00854/) (2024)
- [Two Decades of Bengali Handwritten Digit Recognition: A Survey](https://mlsanity.com/paper/2206.02234/) (2022)
- [MSMatch: Semi-Supervised Multispectral Scene Classification with Few Labels](https://mlsanity.com/paper/2103.10368/) (2021)
- [FAIRER: Fairness as Decision Rationale Alignment](https://mlsanity.com/paper/2306.15299/) (2023)
- [Robust Behavioral Cloning for Autonomous Vehicles using End-to-End Imitation Learning](https://mlsanity.com/paper/2010.04767/) (2020)
- [Hardware Acceleration of Neural Graphics](https://mlsanity.com/paper/2303.05735/) (2023)
- [Platform for Situated Intelligence](https://mlsanity.com/paper/2103.15975/) (2021)
- [Malware Analysis with Artificial Intelligence and a Particular Attention on Results Interpretability](https://mlsanity.com/paper/2107.11100/) (2021)
- [Distributed Adversarial Training to Robustify Deep Neural Networks at Scale](https://mlsanity.com/paper/2206.06257/) (2022)
- [Deep Learning for Medical Image Registration: A Comprehensive Review](https://mlsanity.com/paper/2204.11341/) (2022)
- [CausalDynamics: A large-scale benchmark for structural discovery of dynamical causal models](https://mlsanity.com/paper/2505.16620/) (2025)
- [Marvin: an Innovative Omni-Directional Robotic Assistant for Domestic Environments](https://mlsanity.com/paper/2112.05597/) (2021)
- [Computer vision for liquid samples in hospitals and medical labs using hierarchical image segmentation and relations prediction](https://mlsanity.com/paper/2105.01456/) (2021)
- [Multiscale IoU: A Metric for Evaluation of Salient Object Detection with Fine Structures](https://mlsanity.com/paper/2105.14572/) (2021)
- [Analyzing the Impact of Companies on AI Research Based on Publications](https://mlsanity.com/paper/2310.20444/) (2023)
- [Progressive trajectory matching for medical dataset distillation](https://mlsanity.com/paper/2403.13469/) (2024)
- [CLIP meets DINO for Tuning Zero-Shot Classifier using Unlabeled Image Collections](https://mlsanity.com/paper/2411.19346/) (2024)
- [On the definition and importance of interpretability in scientific machine learning](https://mlsanity.com/paper/2505.13510/) (2025)
- [Towards Efficient Full 8-bit Integer DNN Online Training on Resource-limited Devices without Batch Normalization](https://mlsanity.com/paper/2105.13890/) (2021)
- [SEnSeI: A Deep Learning Module for Creating Sensor Independent Cloud Masks](https://mlsanity.com/paper/2111.08349/) (2021)

## Abstract (arXiv)

Deep learning has demonstrated tremendous success in variety of application domains in the past few years. This new field of machine learning has been growing rapidly and applied in most of the application domains with some new modalities of applications, which helps to open new opportunity. There are different methods have been proposed on different category of learning approaches, which includes supervised, semi-supervised and un-supervised learning. The experimental results show state-of-the-art performance of deep learning over traditional machine learning approaches in the field of Image Processing, Computer Vision, Speech Recognition, Machine Translation, Art, Medical imaging, Medical information processing, Robotics and control, Bio-informatics, Natural Language Processing (NLP), Cyber security, and many more. This report presents a brief survey on development of DL approaches, including Deep Neural Network (DNN), Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) including Long Short Term Memory (LSTM) and Gated Recurrent Units (GRU), Auto-Encoder (AE), Deep Belief Network (DBN), Generative Adversarial Network (GAN), and Deep Reinforcement Learning (DRL). In addition, we have included recent development of proposed advanced variant DL techniques based on the mentioned DL approaches. Furthermore, DL approaches have explored and evaluated in different application domains are also included in this survey. We have also comprised recently developed frameworks, SDKs, and benchmark datasets that are used for implementing and evaluating deep learning approaches. There are some surveys have published on Deep Learning in Neural Networks [1, 38] and a survey on RL [234]. However, those papers have not discussed the individual advanced techniques for training large scale deep learning models and the recently developed method of generative models [1].