mlsanityData to 7 Oct 2026

On the Geometry of Deep Learning

arXiv 2408.04809 ↗First posted 9 Aug 2024cs.LG cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we overview one promising avenue of progress at the mathematical foundation of deep learning: the connection between deep networks and function approximation by affine splines (continuous piecewise linear functions in multiple dimensions). In particular, we will overview work over the past decade on understanding certain geometrical properties of a deep network's affine spline mapping, in particular how it tessellates its input space. As we will see, the affine spline connection and geometrical viewpoint provide a powerful portal through which to view, analyze, and improve the inner workings of a deep network.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 22 references that are in the map. First posted on arXiv 9 Aug 2024.

Builds on

16 of 22 references.

References · 16

  1. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  2. 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 599
  3. 2014On the Number of Linear Regions of Deep Neural Networkscited by 378
  4. 2018Tropical Geometry of Deep Neural Networkscited by 92
  5. 2021What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theorycited by 60
  6. 2018Mad Max: Affine Spline Insights into Deep Learningcited by 54
  7. 2019The Geometry of Deep Networks: Power Diagram Subdivisioncited by 53
  8. 2024Deep Networks Always Grok and Here is Whycited by 49
  9. 2018A representer theorem for deep neural networkscited by 38
  10. 2023SplineCam: Exact Visualization and Characterization of Deep Network Geometry and Decision Boundariescited by 31
  11. 2021MaGNET: Uniform Sampling from Deep Generative Network Manifolds Without Retrainingcited by 23
  12. 2022Polarity Sampling: Quality and Diversity Control of Pre-Trained Generative Networks via Singular Valuescited by 21
4 more
  1. 2018From Hard to Soft: Understanding Deep Network Nonlinearities via Vector Quantization and Statistical Inferencecited by 13
  2. 2020Shallow Univariate ReLu Networks as Splines: Initialization, Loss Surface, Hessian, & Gradient Flow Dynamicscited by 13
  3. 2022Singular Value Perturbation and Deep Network Optimizationcited by 11
  4. 2022Batch Normalization Explainedcited by 8

Built on by

Not cited yet.