mlsanityData to 14 Oct 2026

Axiomatizing Neural Networks via Pursuit of Subspaces

arXiv 2605.20534 ↗First posted 19 May 2026cs.LG cs.AI stat.MLcited by 1

Not on the map. Why?

Abstract (arXiv)

While deep neural networks have achieved remarkable success across a wide range of domains, their underlying mechanisms remain poorly understood, and they are often regarded as black boxes. This gap between empirical performance and theoretical understanding poses a challenge analogous to the pre-axiomatic stage of classical geometry. In this work, we introduce the Pursuit of Subspaces (PoS) hypothesis, an axiomatic framework that formulates neural network behavior through a set of geometric postulates. These axioms, together with their derived consequences, provide a unified perspective on representation, computation, and generalization in both shallow and deep architectures. We show that this framework yields geometric explanations for fundamental questions in deep learning, including representation structure, architectural mechanisms, and generalization behavior, offering a principled step toward a coherent theoretical foundation.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 36 of its 74 references that are in the map. First posted on arXiv 19 May 2026.

Builds on

36 of 74 references.

References · 36

  1. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  4. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  5. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  6. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  7. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  8. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,340
  9. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  10. 2016Group Equivariant Convolutional Networkscited by 1,165
  11. 2021CvT: Introducing Convolutions to Vision Transformerscited by 947
  12. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
24 more
  1. 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 629
  2. 2014On the Number of Linear Regions of Deep Neural Networkscited by 381
  3. 2016On the Expressive Power of Deep Neural Networkscited by 306
  4. 2018On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groupscited by 300
  5. 2019Gauge Equivariant Convolutional Networks and the Icosahedral CNNcited by 267
  6. 2016Variational Lossy Autoencodercited by 176
  7. 2017Bounding and Counting Linear Regions of Deep Neural Networkscited by 161
  8. 2017Towards Robust Neural Networks via Random Self-ensemblecited by 159
  9. 2015Adding Gradient Noise Improves Learning for Very Deep Networkscited by 145
  10. 2019Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Groupcited by 134
  11. 2013On the number of response regions of deep feed forward networks with piece-wise linear activationscited by 94
  12. 2019Orthogonal Deep Neural Networkscited by 72
  13. 2022Masked Frequency Modeling for Self-Supervised Visual Pre-Trainingcited by 71
  14. 2020The Unreasonable Effectiveness of Deep Learning in Artificial Intelligencecited by 57
  15. 2024Learning by Reconstruction Produces Uninformative Features For Perceptioncited by 55
  16. 2021Neural population geometry: An approach for understanding biological and artificial neural networkscited by 52
  17. 2024Masked Autoencoders for Microscopy are Scalable Learners of Cellular Biologycited by 52
  18. 2021Neural tuning and representational geometrycited by 31
  19. 2023SplineCam: Exact Visualization and Characterization of Deep Network Geometry and Decision Boundariescited by 31
  20. 2023Mortal Computation: A Foundation for Biomimetic Intelligencecited by 15
  21. 2021Robust Peak Detection for Holter ECGs by Self-Organized Operational Neural Networkscited by 11
  22. 2022A Personalized Zero-Shot ECG Arrhythmia Monitoring System: From Sparse Representation Based Domain Adaption to Energy Efficient Abnormal Beat Detection for Practical ECG Surveillancecited by 6
  23. 2020Convolutional Sparse Support Estimator Network (CSEN) From energy efficient support estimation to learning-aided Compressive Sensingcited by 5
  24. 2023Operational Support Estimator Networkscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026When Does Unsupervised Learning Succeed or Fail? A PoS Perspective on Reconstruction-Based Anomaly Detectioncited by 0