mlsanityData to 14 Oct 2026

Deep Learning as the Disciplined Construction of Tame Objects

arXiv 2509.18025 ↗First posted 22 Sep 2025math.OC cs.AI cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

One can see deep-learning models as compositions of functions within the so-called tame geometry. In this expository note, we give an overview of some topics at the interface of tame geometry (also known as o-minimality), optimization theory, and deep learning theory and practice. To do so, we gradually introduce the concepts and tools used to build convergence guarantees for stochastic gradient descent in a general nonsmooth nonconvex, but tame, setting. This illustrates some ways in which tame geometry is a natural mathematical framework for the study of AI systems, especially within Deep Learning.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 21 of its 117 references that are in the map. First posted on arXiv 22 Sep 2025.

Builds on

21 of 117 references.

References · 21

  1. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  2. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,150
  3. 2021Activation Functions in Deep Learning: A Comprehensive Survey and Benchmarkcited by 155
  4. 2018Activation Functions: Comparison of trends in Practice and Research for Deep Learningcited by 130
  5. 2018Stochastic subgradient method converges on tame functionscited by 124
  6. 2019Conservative set valued fields, automatic differentiation, stochastic gradient method and deep learningcited by 72
  7. 2017Stochastic Methods for Composite and Weakly Convex Optimization Problemscited by 48
  8. 2020A mathematical model for automatic differentiation in machine learningcited by 45
  9. 2020Convergence of constant step stochastic gradient descent for non-smooth non-convex functionscited by 27
  10. 2024Three Decades of Activations: A Comprehensive Survey of 400 Activation Functions for Neural Networkscited by 23
  11. 2020On Correctness of Automatic Differentiation for Non-Differentiable Functionscited by 20
  12. 2021Active manifolds, stratifications, and convergence to local minima in nonsmooth optimizationcited by 11
9 more
  1. 2018Provably Correct Automatic Subdifferentiation for Qualified Programscited by 8
  2. 2020Curiosities and counterexamples in smooth convex optimizationcited by 7
  3. 2020Newton acceleration on manifolds identified by proximal-gradient methodscited by 5
  4. 2025SAD Neural Networks: Divergent Gradient Flows and Asymptotic Optimality via o-minimal Structurescited by 5
  5. 2021The structure of conservative gradient fieldscited by 4
  6. 2024Convergence of Descent Optimization Algorithms under Polyak-L ojasiewicz-Kurdyka Conditionscited by 4
  7. 2024Measurability in the Fundamental Theorem of Statistical Learningcited by 4
  8. 2021Certifying Global Optimality of AC-OPF Solutions via sparse polynomial optimizationcited by 1
  9. 2023Piecewise Polynomial Regression of Tame Functions via Integer Programmingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Stochastic Penalty-Barrier Method for Constrained Machine Learningcited by 0
  2. 2026Fast approximation and learning of binary classification tasks in o-minimal structures using ReLU neural networkscited by 0
  3. 2026Lipschitzian SLLNs for random functionscited by 0