# A Review of Deep Learning with Special Emphasis on Architectures, Applications and Recent Trends

arXiv: https://arxiv.org/abs/1905.13294 · HTML: https://mlsanity.com/paper/1905.13294/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 May 2019.

## Built on by (most cited first, 26 of 26)

- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [A Review on Methods and Applications in Multimodal Deep Learning](https://mlsanity.com/paper/2202.09195/) (2022)
- [Recent Advances and Trends in Multimodal Deep Learning: A Review](https://mlsanity.com/paper/2105.11087/) (2021)
- [Generative Dataset Distillation: Balancing Global Structure and Local Details](https://mlsanity.com/paper/2404.17732/) (2024)
- [Supervised training of spiking neural networks for robust deployment on mixed-signal neuromorphic processors](https://mlsanity.com/paper/2102.06408/) (2021)
- [A Critical Study on the Recent Deep Learning Based Semi-Supervised Video Anomaly Detection Methods](https://mlsanity.com/paper/2111.01604/) (2021)
- [Prediction and Uncertainty Quantification of SAFARI-1 Axial Neutron Flux Profiles with Neural Networks](https://mlsanity.com/paper/2211.08654/) (2022)
- [Hitchhiker's guide on the relation of Energy-Based Models with other generative models, sampling and statistical physics: a comprehensive review](https://mlsanity.com/paper/2406.13661/) (2024)
- [Computational Models to Study Language Processing in the Human Brain: A Survey](https://mlsanity.com/paper/2403.13368/) (2024)
- [Gi and Pal Scores: Deep Neural Network Generalization Statistics](https://mlsanity.com/paper/2104.03469/) (2021)
- [Stochastic Modeling of Inhomogeneities in the Aortic Wall and Uncertainty Quantification using a Bayesian Encoder-Decoder Surrogate](https://mlsanity.com/paper/2202.10244/) (2022)
- [GAS-Norm: Score-Driven Adaptive Normalization for Non-Stationary Time Series Forecasting in Deep Learning](https://mlsanity.com/paper/2410.03935/) (2024)
- [PINN - a Domain Decomposition Method for Bayesian Physics-Informed Neural Networks](https://mlsanity.com/paper/2504.19013/) (2025)
- [Emulating Quantum Dynamics with Neural Networks via Knowledge Distillation](https://mlsanity.com/paper/2203.10200/) (2022)
- [Fraud Analytics: A Decade of Research -- Organizing Challenges and Solutions in the Field](https://mlsanity.com/paper/2212.04329/) (2022)
- [Clustering and Uncertainty Analysis to Improve the Machine Learning-based Predictions of SAFARI-1 Control Follower Assembly Axial Neutron Flux Profiles](https://mlsanity.com/paper/2312.14193/) (2023)
- [Advances in Electron Microscopy with Deep Learning](https://mlsanity.com/paper/2101.01178/) (2021)
- [Energy time series forecasting-Analytical and empirical assessment of conventional and machine learning models](https://mlsanity.com/paper/2108.10663/) (2021)
- [An Artificial Neural Network Functionalized by Evolution](https://mlsanity.com/paper/2205.10118/) (2022)
- [Generalization ability and Vulnerabilities to adversarial perturbations: Two sides of the same coin](https://mlsanity.com/paper/2205.10952/) (2022)
- [Survey on Deep Fuzzy Systems in regression applications: a view on interpretability](https://mlsanity.com/paper/2209.04230/) (2022)
- [Scientific Impact of Graph-Based Approaches in Deep Learning Studies -- A Bibliometric Comparison](https://mlsanity.com/paper/2210.07343/) (2022)
- [Having Second Thoughts? Let's hear it](https://mlsanity.com/paper/2311.15356/) (2023)
- [The State of Lithium-Ion Battery Health Prognostics in the CPS Era](https://mlsanity.com/paper/2403.19816/) (2024)
- [Searching for internal symbols underlying deep learning](https://mlsanity.com/paper/2405.20605/) (2024)
- [Two out of Three (ToT): using self-consistency to make robust predictions](https://mlsanity.com/paper/2505.12642/) (2025)

## Abstract (arXiv)

Deep learning has solved a problem that as little as five years ago was thought by many to be intractable - the automatic recognition of patterns in data; and it can do so with accuracy that often surpasses human beings. It has solved problems beyond the realm of traditional, hand-crafted machine learning algorithms and captured the imagination of practitioners trying to make sense out of the flood of data that now inundates our society. As public awareness of the efficacy of DL increases so does the desire to make use of it. But even for highly trained professionals it can be daunting to approach the rapidly increasing body of knowledge produced by experts in the field. Where does one start? How does one determine if a particular model is applicable to their problem? How does one train and deploy such a network? A primer on the subject can be a good place to start. With that in mind, we present an overview of some of the key multilayer ANNs that comprise DL. We also discuss some new automatic architecture optimization protocols that use multi-agent approaches. Further, since guaranteeing system uptime is becoming critical to many computer applications, we include a section on using neural networks for fault detection and subsequent mitigation. This is followed by an exploratory survey of several application areas where DL has emerged as a game-changing technology: anomalous behavior detection in financial applications or in financial time-series forecasting, predictive and prescriptive analytics, medical image processing and analysis and power systems research. The thrust of this review is to outline emerging areas of application-oriented research within the DL community as well as to provide a reference to researchers seeking to use it in their work for what it does best: statistical pattern recognition with unparalleled learning capacity with the ability to scale with information.