# Annotated History of Modern AI and Deep Learning

arXiv: https://arxiv.org/abs/2212.11279 · HTML: https://mlsanity.com/paper/2212.11279/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Dec 2022.

## Built on by (most cited first, 20 of 20)

- [A Comprehensive Survey of Deep Transfer Learning for Anomaly Detection in Industrial Time Series: Methods, Applications, and Directions](https://mlsanity.com/paper/2307.05638/) (2023)
- [Self-Improvements in Modern Agentic Systems: A Survey](https://mlsanity.com/paper/2607.13104/) (2026)
- [Learning to Identify Critical States for Reinforcement Learning from Videos](https://mlsanity.com/paper/2308.07795/) (2023)
- [Dissociating Artificial Intelligence from Artificial Consciousness](https://mlsanity.com/paper/2412.04571/) (2024)
- [Quantum Vision Transformers for Quark-Gluon Classification](https://mlsanity.com/paper/2405.10284/) (2024)
- [Thinking Beyond Tokens: From Brain-Inspired Intelligence to Cognitive Foundations for Artificial General Intelligence and its Societal Impact](https://mlsanity.com/paper/2507.00951/) (2025)
- [Topology meets Machine Learning: An Introduction using the Euler Characteristic Transform](https://mlsanity.com/paper/2410.17760/) (2024)
- [The Deep Arbitrary Polynomial Chaos Neural Network or how Deep Artificial Neural Networks could benefit from Data-Driven Homogeneous Chaos Theory](https://mlsanity.com/paper/2306.14753/) (2023)
- [Eternal Sunshine of the Mechanical Mind: The Irreconcilability of Machine Learning and the Right to be Forgotten](https://mlsanity.com/paper/2403.05592/) (2024)
- [LayerShuffle: Enhancing Robustness in Vision Transformers by Randomizing Layer Execution Order](https://mlsanity.com/paper/2407.04513/) (2024)
- [AI Generations: From AI 1.0 to AI 4.0](https://mlsanity.com/paper/2502.11312/) (2025)
- [Development of Hybrid Artificial Intelligence Training on Real and Synthetic Data: Benchmark on Two Mixed Training Strategies](https://mlsanity.com/paper/2506.24093/) (2025)
- [Building artificial neural circuits for domain-general cognition: a primer on brain-inspired systems-level architecture](https://mlsanity.com/paper/2303.13651/) (2023)
- [LayerCollapse: Adaptive compression of neural networks](https://mlsanity.com/paper/2311.17943/) (2023)
- [Cross Entropy in Deep Learning of Classifiers Is Unnecessary -- ISBE Error is All You Need](https://mlsanity.com/paper/2311.16357/) (2023)
- [Deep Learning 2.0: Artificial Neurons That Matter -- Reject Correlation, Embrace Orthogonality](https://mlsanity.com/paper/2411.08085/) (2024)
- [A Statistical Framework for Model Selection in LSTM Networks](https://mlsanity.com/paper/2506.06840/) (2025)
- [Match Chat: Real Time Generative AI and Generative Computing for Tennis](https://mlsanity.com/paper/2509.12592/) (2025)
- [Who invented deep residual learning?](https://mlsanity.com/paper/2509.24732/) (2025)
- [Artificially intelligent agents in the social and behavioral sciences: A history and outlook](https://mlsanity.com/paper/2510.05743/) (2025)

## Abstract (arXiv)

Machine learning (ML) is the science of credit assignment. It seeks to find patterns in observations that explain and predict the consequences of events and actions. This then helps to improve future performance. Minsky's so-called "fundamental credit assignment problem" (1963) surfaces in all sciences including physics (why is the world the way it is?) and history (which persons/ideas/actions have shaped society and civilisation?). Here I focus on the history of ML itself. Modern artificial intelligence (AI) is dominated by artificial neural networks (NNs) and deep learning, both of which are conceptually closer to the old field of cybernetics than what was traditionally called AI (e.g., expert systems and logic programming). A modern history of AI & ML must emphasize breakthroughs outside the scope of shallow AI text books. In particular, it must cover the mathematical foundations of today's NNs such as the chain rule (1676), the first NNs (circa 1800), the first practical AI (1914), the theory of AI and its limitations (1931-34), and the first working deep learning algorithms (1965-). From the perspective of 2025, I provide a timeline of the most significant events in the history of NNs, ML, deep learning, AI, computer science, and mathematics in general, crediting the individuals who laid the field's foundations. The text contains numerous hyperlinks to relevant overview sites. With a ten-year delay, it supplements my 2015 award-winning deep learning survey which provides hundreds of additional references. Finally, I will put things in a broader historical context, spanning from the Big Bang to when the universe will be many times older than it is now.