mlsanityData to 14 Oct 2026

The Mathematics of AI Winters: The mathematical Taxonomy of Paradigm Fragility in AI Winter

arXiv 2606.12610 ↗First posted 10 Jun 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Two major periods of reduced funding and confidence in artificial intelligence research, commonly called the first and second AI winters, are usually explained through engineering failure, commercial disappointment, and inflated expectations. This article develops a complementary thesis: that the dominant paradigms of those periods also met genuine formal barriers, including limitations of representation, optimisation, computational complexity, statistical learnability, and high-dimensional approximation. The contribution is synthetic rather than archival. We do not claim that particular theorems mechanically caused the winters; rather, we show that several central disappointments of early AI were aligned with mathematically precise bottlenecks. We analyse these bottlenecks through the perceptron impossibility results of Minsky and Papert, the complexity-theoretic hardness of exact neural-network training established by Blum and Rivest, minimax rates for nonparametric estimation in high dimension due to Stone, vanishing-gradient analyses by Hochreiter and by Bengio and collaborators, and classical statistical learning theory in the tradition of Vapnik and Chervonenkis, Valiant, and Blumer and collaborators. We then relate these barriers to the later breakthroughs that mitigated, rather than eliminated, them.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 56 references that are in the map. First posted on arXiv 10 Jun 2026.

Builds on

19 of 56 references.

References · 19

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Scaling Laws for Neural Language Modelscited by 6,586
  3. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  4. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  5. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  6. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  7. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  8. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
  9. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 719
  10. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 655
  11. 2017Exploring Generalization in Deep Learningcited by 600
  12. 2019Surprises in High-Dimensional Ridgeless Least Squares Interpolationcited by 513
7 more
  1. 2014On the Number of Linear Regions of Deep Neural Networkscited by 381
  2. 2016Exponential expressivity in deep neural networks through transient chaoscited by 261
  3. 2018Implicit Bias of Gradient Descent on Linear Convolutional Networkscited by 251
  4. 2016Benefits of depth in neural networkscited by 250
  5. 2021Deep learning: a statistical viewpointcited by 240
  6. 2015The Power of Depth for Feedforward Neural Networkscited by 183
  7. 2015The Optimal Sample Complexity of PAC Learningcited by 72

Built on by

Not cited yet.