mlsanityData to 7 Oct 2026

Deep Learning in Neural Networks: An Overview

arXiv 1404.7828 ↗First posted 30 Apr 2014cs.NE cs.LGcited by 765

Not on the map. Why?

Abstract (arXiv)

In recent years, deep artificial neural networks (including recurrent ones) have won numerous contests in pattern recognition and machine learning. This historical survey compactly summarises relevant work, much of it from the previous millennium. Shallow and deep learners are distinguished by the depth of their credit assignment paths, which are chains of possibly learnable, causal links between actions and effects. I review deep supervised learning (also recapitulating the history of backpropagation), unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks.

Key facts. Cited by 765 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Apr 2014.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 210; 2022: 170; 2023: 151; 2024: 123; 2025: 81; 2026: 30.

765 papers.

Sort:

By area:

  1. 2021Deep Neural Networks and Tabular Data: A Surveycited by 489
  2. 2021ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biasescited by 484
  3. 2021Escaping the Big Data Paradigm with Compact Transformerscited by 261
  4. 2021Geometric and Physical Quantities Improve E(3) Equivariant Message Passingcited by 225
  5. 2021Dash: Semi-Supervised Learning with Dynamic Thresholdingcited by 147
  6. 2023End-to-End Speech Recognition: A Surveycited by 127
  7. 2022EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applicationscited by 121
  8. 2021Geometric Deep Learning on Molecular Representationscited by 113
  9. 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 105
  10. 2021Filling the G_ap_s: Multivariate Time Series Imputation by Graph Neural Networkscited by 93
  11. 2021Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methodscited by 83
  12. 2024Language Agents as Optimizable Graphscited by 81
38 more
  1. 2022Knowledge Tracing: A Surveycited by 76
  2. 2022Deep Learning in Multimodal Remote Sensing Data Fusion: A Comprehensive Reviewcited by 73
  3. 2022ComENet: Towards Complete and Efficient Message Passing for 3D Molecular Graphscited by 69
  4. 2023Deep Learning for Time Series Classification and Extrinsic Regression: A Current Surveycited by 68
  5. 2021Multi-Modal 3D Object Detection in Autonomous Driving: a Surveycited by 63
  6. 2023Computationally Budgeted Continual Learning: What Does Matter?cited by 61
  7. 2021Visual Representation Learning Does Not Generalize Strongly Within the Same Domaincited by 60
  8. 2023Improving Visual Prompt Tuning for Self-supervised Vision Transformerscited by 57
  9. 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
  10. 2024MarDini: Masked Autoregressive Diffusion for Video Generation at Scalecited by 49
  11. 2021GLocalX -- From Local to Global Explanations of Black Box AI Modelscited by 48
  12. 2021An Experimental Review on Deep Learning Architectures for Time Series Forecastingcited by 47
  13. 2023Real-Time Evaluation in Online Continual Learning: A New Hopecited by 47
  14. 2023RealGen: Retrieval Augmented Generation for Controllable Traffic Scenarioscited by 46
  15. 2023A Transformer-based representation-learning model with unified processing of multimodal input for clinical diagnosticscited by 42
  16. 2021Recent Advances on Neural Network Pruning at Initializationcited by 41
  17. 2023Image Shortcut Squeezing: Countering Perturbative Availability Poisons with Compressioncited by 39
  18. 2023When Deep Learning Meets Polyhedral Theory: A Surveycited by 39
  19. 2021Physics and Equality Constrained Artificial Neural Networks: Application to Forward and Inverse Problems with Multi-fidelity Data Fusioncited by 38
  20. 2022Improving Molecular Contrastive Learning via Faulty Negative Mitigation and Decomposed Fragment Contrastcited by 37
  21. 2022A Law of Data Separation in Deep Learningcited by 36
  22. 2021Fast Axiomatic Attribution for Neural Networkscited by 35
  23. 2021How to avoid machine learning pitfalls: a guide for academic researcherscited by 34
  24. 2022A mixed formulation for physics-informed neural networks as a potential solver for engineering problems in heterogeneous domains: comparison with finite element methodcited by 33
  25. 2024Suitability of KANs for Computer Vision: A preliminary investigationcited by 33
  26. 2024Quantum Convolutional Neural Networks are Effectively Classically Simulablecited by 33
  27. 2021A Review on Deep Learning in UAV Remote Sensingcited by 32
  28. 2023A Survey on Dataset Distillation: Approaches, Applications and Future Directionscited by 32
  29. 2021On the Existence of Universal Lottery Ticketscited by 30
  30. 2022Data-driven emergence of convolutional structure in neural networkscited by 29
  31. 2022Diffeomorphic Counterfactuals with Generative Modelscited by 29
  32. 2022A review of predictive uncertainty estimation with machine learningcited by 29
  33. 2023Tensor Networks Meet Neural Networks: A Survey and Future Perspectivescited by 29
  34. 2021A Deep Learning approach to Reduced Order Modelling of Parameter Dependent Partial Differential Equationscited by 28
  35. 2022Machine Learning for Microcontroller-Class Hardware: A Reviewcited by 28
  36. 2022Ab-initio quantum chemistry with neural-network wavefunctionscited by 28
  37. 2021A Survey on Anomaly Detection for Technical Systems using LSTM Networkscited by 27
  38. 2021Kernel Continual Learningcited by 27

50 most cited of 765 All 765 (JSON)