Defining Neural Network Architecture through Polytope Structures of Dataset
Not on the map. Why?
Abstract (arXiv)
Current theoretical and empirical research in neural networks suggests that complex datasets require large network architectures for thorough classification, yet the precise nature of this relationship remains unclear. This paper tackles this issue by defining upper and lower bounds for neural network widths, which are informed by the polytope structure of the dataset in question. We also delve into the application of these principles to simplicial complexes and specific manifold shapes, explaining how the requirement for network width varies in accordance with the geometric complexity of the dataset. Moreover, we develop an algorithm to investigate a converse situation where the polytope structure of a dataset can be inferred from its corresponding trained neural networks. Through our algorithm, it is established that popular datasets such as MNIST, Fashion-MNIST, and CIFAR10 can be efficiently encapsulated using no more than two polytopes with a small number of faces.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 48 of its 69 references that are in the map. First posted on arXiv 4 Feb 2024.
Builds on
48 of 69 references.
References · 48
- 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
- 2020Extracting Training Data from Large Language Modelscited by 2,144
- 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
- 2016Input Convex Neural Networkscited by 407
- 2018Measuring the Intrinsic Dimension of Objective Landscapescited by 364
- 2019Deconstructing Lottery Tickets: Zeros, Signs, and the Supermaskcited by 206
- 2019Optimal transport mapping via input convex neural networkscited by 202
- 2020Proving the Lottery Ticket Hypothesis: Pruning is All You Needcited by 200
- 2018Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balancedcited by 172
- 2019Deep ReLU Networks Have Surprisingly Few Activation Patternscited by 160
- 2022Reconstructing Training Data from Trained Neural Networkscited by 129
- 2020Topology of deep neural networkscited by 111
36 more
- 2020Minimum Width for Universal Approximationcited by 92
- 2015Representation Benefits of Deep Feedforward Networkscited by 84
- 2019Reverse-Engineering Deep ReLU Networkscited by 81
- 2019Nonparametric Regression on Low-Dimensional Manifolds using Deep ReLU Networks : Function Approximation and Statistical Recoverycited by 73
- 2021Intrinsic Dimension, Persistent Homology and Generalization in Neural Networkscited by 70
- 2021Representation Topology Divergence: A Method for Comparing Neural Network Representationscited by 66
- 2018Blessing of dimensionality: mathematical foundations of the statistical physics of datacited by 55
- 2020Scalable Computations of Wasserstein Barycenter via Input Convex Neural Networkscited by 49
- 2023When Deep Learning Meets Polyhedral Theory: A Surveycited by 39
- 2023Lower Bounds on the Depth of Integral ReLU Neural Networks via Lattice Polytopescited by 36
- 2020Deep Networks and the Multiple Manifold Problemcited by 35
- 2020On transversality of bent hyperplane arrangements and the topological expressiveness of ReLU neural networkscited by 34
- 2022Interpreting Neural Networks through the Polytope Lenscited by 33
- 2021Manifold Topology Divergence: a Framework for Comparing Data Manifoldscited by 31
- 2020On the Decision Boundaries of Neural Networks: A Tropical Geometry Perspectivecited by 25
- 2022Algorithmic Determination of the Combinatorial Structure of the Linear Regions of ReLU Neural Networkscited by 21
- 2020Input Convex Neural Networks for Building MPCcited by 17
- 2020The Curious Case of Convex Neural Networkscited by 16
- 2021The Separation Capacity of Random Neural Networkscited by 14
- 2023Polyhedral Complex Extraction from ReLU Networks using Edge Subdivisioncited by 14
- 2021Traversing the Local Polytopes of ReLU Neural Networks: A Unified Approach for Network Verificationcited by 13
- 2022Topology and geometry of data manifold in deep learningcited by 12
- 2022Effects of Data Geometry in Early Deep Learningcited by 12
- 2023Exact Count of Boundary Pieces of ReLU Classifiers: Towards the Proper Complexity Measure for Classificationcited by 11
- 2023ReLU Neural Networks, Polyhedral Decompositions, and Persistent Homologcited by 10
- 2020Input Convex Neural Networks for Optimal Voltage Regulationcited by 7
- 2021CDiNN -Convex Difference Neural Networkscited by 6
- 2021Deep Networks Provably Classify Data on Curvescited by 6
- 2022Reachable Polyhedral Marching (RPM): An Exact Analysis Tool for Deep-Learned Control Systemscited by 6
- 2018On decision regions of narrow deep neural networkscited by 5
- 2020A Topological Framework for Deep Learningcited by 5
- 2021Topological Deep Learning: Classification Neural Networkscited by 5
- 2023Deep ReLU Networks Have Surprisingly Simple Polytopescited by 5
- 2019Geometry of Deep Convolutional Networkscited by 4
- 2023The Geometric Structure of Fully-Connected ReLU Layerscited by 4
- 2023Topology Estimation of Simulated 4D Image Data by Combining Downscaling and Convolutional Neural Networkscited by 2
Built on by
Citing papers per year: 2024: 1; 2025: 1.
2 papers.