mlsanityData to 7 Oct 2026

On Functional Dimension and Persistent Pseudodimension

arXiv 2410.17191 ↗First posted 22 Oct 2024cs.LG math.AG math.COcited by 1

Not on the map. Why?

Abstract (arXiv)

For any fixed feedforward ReLU neural network architecture, it is well-known that many different parameter settings can determine the same function. It is less well-known that the degree of this redundancy is inhomogeneous across parameter space. In this work, we discuss two locally applicable complexity measures for ReLU network classes and what we know about the relationship between them: (1) the local functional dimension [14, 18], and (2) a local version of VC dimension that we call persistent pseudodimension. The former is easy to compute on finite batches of points; the latter should give local bounds on the generalization gap, which would inform an understanding of the mechanics of the double descent phenomenon [7].

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 15 of its 35 references that are in the map. First posted on arXiv 22 Oct 2024.

Builds on

15 of 35 references.

References · 15

  1. 2019The generalization error of random features regression: Precise asymptotics and double descent curvecited by 410
  2. 2017Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networkscited by 304
  3. 2016Understanding Deep Neural Networks with Rectified Linear Unitscited by 253
  4. 2021Deep learning: a statistical viewpointcited by 239
  5. 2019Deep ReLU Networks Have Surprisingly Few Activation Patternscited by 160
  6. 2018Tropical Geometry of Deep Neural Networkscited by 92
  7. 2019Reverse-Engineering Deep ReLU Networkscited by 81
  8. 2021The Modern Mathematics of Deep Learningcited by 63
  9. 2023Hidden symmetries of ReLU networkscited by 38
  10. 2020On transversality of bent hyperplane arrangements and the topological expressiveness of ReLU neural networkscited by 34
  11. 2022Algorithmic Determination of the Combinatorial Structure of the Linear Regions of ReLU Neural Networkscited by 21
  12. 2021An Embedding of ReLU Networks and an Analysis of their Identifiabilitycited by 20
3 more
  1. 2022Functional dimension of feedforward ReLU neural networkscited by 20
  2. 2022Local Identifiability of Deep ReLU Neural Networks: the Theorycited by 12
  3. 2024Geometry-induced Regularization in Deep ReLU Neural Networkscited by 1

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Regularization Implies balancedness in the deep linear networkcited by 4