# Deep Learning in Neural Networks: An Overview

arXiv: https://arxiv.org/abs/1404.7828 · HTML: https://mlsanity.com/paper/1404.7828/

## Key facts

- Cited by 765 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 30 Apr 2014.

## Built on by (most cited first, 30 of 765)

- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases](https://mlsanity.com/paper/2103.10697/) (2021)
- [Escaping the Big Data Paradigm with Compact Transformers](https://mlsanity.com/paper/2104.05704/) (2021)
- [Geometric and Physical Quantities Improve E(3) Equivariant Message Passing](https://mlsanity.com/paper/2110.02905/) (2021)
- [Dash: Semi-Supervised Learning with Dynamic Thresholding](https://mlsanity.com/paper/2109.00650/) (2021)
- [End-to-End Speech Recognition: A Survey](https://mlsanity.com/paper/2303.03329/) (2023)
- [EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applications](https://mlsanity.com/paper/2206.10589/) (2022)
- [Geometric Deep Learning on Molecular Representations](https://mlsanity.com/paper/2107.12375/) (2021)
- [Nonparametric Identifiability of Causal Representations from Unknown Interventions](https://mlsanity.com/paper/2306.00542/) (2023)
- [Filling the G_ap_s: Multivariate Time Series Imputation by Graph Neural Networks](https://mlsanity.com/paper/2108.00298/) (2021)
- [Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methods](https://mlsanity.com/paper/2111.02398/) (2021)
- [Language Agents as Optimizable Graphs](https://mlsanity.com/paper/2402.16823/) (2024)
- [Knowledge Tracing: A Survey](https://mlsanity.com/paper/2201.06953/) (2022)
- [Deep Learning in Multimodal Remote Sensing Data Fusion: A Comprehensive Review](https://mlsanity.com/paper/2205.01380/) (2022)
- [ComENet: Towards Complete and Efficient Message Passing for 3D Molecular Graphs](https://mlsanity.com/paper/2206.08515/) (2022)
- [Deep Learning for Time Series Classification and Extrinsic Regression: A Current Survey](https://mlsanity.com/paper/2302.02515/) (2023)
- [Multi-Modal 3D Object Detection in Autonomous Driving: a Survey](https://mlsanity.com/paper/2106.12735/) (2021)
- [Computationally Budgeted Continual Learning: What Does Matter?](https://mlsanity.com/paper/2303.11165/) (2023)
- [Visual Representation Learning Does Not Generalize Strongly Within the Same Domain](https://mlsanity.com/paper/2107.08221/) (2021)
- [Improving Visual Prompt Tuning for Self-supervised Vision Transformers](https://mlsanity.com/paper/2306.05067/) (2023)
- [Semantics for Robotic Mapping, Perception and Interaction: A Survey](https://mlsanity.com/paper/2101.00443/) (2021)
- [MarDini: Masked Autoregressive Diffusion for Video Generation at Scale](https://mlsanity.com/paper/2410.20280/) (2024)
- [GLocalX -- From Local to Global Explanations of Black Box AI Models](https://mlsanity.com/paper/2101.07685/) (2021)
- [An Experimental Review on Deep Learning Architectures for Time Series Forecasting](https://mlsanity.com/paper/2103.12057/) (2021)
- [Real-Time Evaluation in Online Continual Learning: A New Hope](https://mlsanity.com/paper/2302.01047/) (2023)
- [RealGen: Retrieval Augmented Generation for Controllable Traffic Scenarios](https://mlsanity.com/paper/2312.13303/) (2023)
- [A Transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics](https://mlsanity.com/paper/2306.00864/) (2023)
- [Recent Advances on Neural Network Pruning at Initialization](https://mlsanity.com/paper/2103.06460/) (2021)
- [Image Shortcut Squeezing: Countering Perturbative Availability Poisons with Compression](https://mlsanity.com/paper/2301.13838/) (2023)
- [When Deep Learning Meets Polyhedral Theory: A Survey](https://mlsanity.com/paper/2305.00241/) (2023)

## Abstract (arXiv)

In recent years, deep artificial neural networks (including recurrent ones) have won numerous contests in pattern recognition and machine learning. This historical survey compactly summarises relevant work, much of it from the previous millennium. Shallow and deep learners are distinguished by the depth of their credit assignment paths, which are chains of possibly learnable, causal links between actions and effects. I review deep supervised learning (also recapitulating the history of backpropagation), unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks.