mlsanityData to 14 Oct 2026

Fractal and Regular Geometry of Deep Neural Networks

arXiv 2504.06250 ↗First posted 8 Apr 2025math.PR cs.LG stat.MLcited by 1

Not on the map. Why?

Abstract (arXiv)

We study the geometric properties of random neural networks by investigating the boundary volumes of their excursion sets for different activation functions, as the depth increases. More specifically, we show that, for activations which are not very regular (e.g., the Heaviside step function), the boundary volumes exhibit fractal behavior, with their Hausdorff dimension monotonically increasing with the depth. On the other hand, for activations which are more regular (e.g., ReLU, logistic and tanh), as the depth increases, the expected boundary volumes can either converge to zero, remain constant or diverge exponentially, depending on a single spectral parameter which can be easily computed. Our theoretical results are confirmed in some numerical experiments based on Monte Carlo simulations.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 16 of its 51 references that are in the map. First posted on arXiv 8 Apr 2025.

Builds on

16 of 51 references.

References · 16

  1. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 787
  2. 2017Deep Neural Networks as Gaussian Processescited by 622
  3. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 433
  4. 2018On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transportcited by 426
  5. 2014On the Number of Linear Regions of Deep Neural Networkscited by 381
  6. 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 317
  7. 2021Deep learning: a statistical viewpointcited by 240
  8. 2019Complexity of Linear Regions in Deep Networkscited by 167
  9. 2020Deep Equals Shallow for ReLU Networks in Kernel Regimescited by 85
  10. 2021Random Neural Networks in the Infinite Width Limit as Gaussian Processescited by 44
  11. 2022Quantitative Gaussian Approximation of Randomly Initialized Deep Neural Networkscited by 25
  12. 2022On the Number of Regions of Piecewise Linear Neural Networkscited by 21
4 more
  1. 2023Quantitative CLTs in Deep Neural Networkscited by 20
  2. 2023A Quantitative Functional Central Limit Theorem for Shallow Neural Networkscited by 8
  3. 2021Measuring Complexity of Learning Schemes Using Hessian-Schatten Total Variationcited by 4
  4. 2024Spectral complexity of deep neural networkscited by 3

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Critical Points of Random Neural Networkscited by 0