mlsanityData to 14 Oct 2026

Metric Space Magnitude and Generalisation in Neural Networks

arXiv 2305.05611 ↗First posted 9 May 2023cs.LG math.GT stat.MLcited by 15

Not on the map. Why?

Abstract (arXiv)

Deep learning models have seen significant successes in numerous applications, but their inner workings remain elusive. The purpose of this work is to quantify the learning process of deep neural networks through the lens of a novel topological invariant called magnitude. Magnitude is an isometry invariant; its properties are an active area of research as it encodes many known invariants of a metric space. We use magnitude to study the internal representations of neural networks and propose a new method for determining their generalisation capabilities. Moreover, we theoretically connect magnitude dimension and the generalisation error, and demonstrate experimentally that the proposed framework can be a good indicator of the latter.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 14 of its 39 references that are in the map. First posted on arXiv 9 May 2023.

Builds on

14 of 39 references.

References · 14

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
  3. 2019A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networkscited by 214
  4. 2019Traditional and Heavy-Tailed Self Regularization in Neural Network Modelscited by 105
  5. 2022Topological Deep Learning: Going Beyond Graph Datacited by 97
  6. 2019A Topology Layer for Machine Learningcited by 80
  7. 2018Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topologycited by 73
  8. 2021Intrinsic Dimension, Persistent Homology and Generalization in Neural Networkscited by 70
  9. 2020Hausdorff Dimension, Heavy Tails, and Generalization in Neural Networkscited by 62
  10. 2018Persistent Homology and the Upper Box Dimensioncited by 15
  11. 2022Topology and geometry of data manifold in deep learningcited by 12
  12. 2021Weighting vectors for machine learning: numerical harmonic analysis applied to boundary detectioncited by 6
2 more
  1. 2021The magnitude vector of imagescited by 4
  2. 2019A Primer on Persistent Homology of Finite Metric Spacescited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 3; 2025: 5; 2026: 5.

15 papers.

Sort:

  1. 2024Position: Topological Deep Learning is the New Frontier for Relational Learningcited by 66
  2. 2023Metric Space Magnitude for Evaluating the Diversity of Latent Representationscited by 17
  3. 2024The Effect of Intrinsic Dataset Properties on Generalization: Unraveling Learning Differences Between Natural and Medical Imagescited by 13
  4. 2023Topological Data Analysis for Neural Network Analysis: A Comprehensive Surveycited by 9
  5. 2025Geometry-Aware Edge Pooling for Graph Neural Networkscited by 5
  6. 2024Approximating Metric Magnitude of Point Setscited by 4
  7. 2026Magnitude Distance: A Geometric Measure of Dataset Similaritycited by 2
  8. 2025Is magnitude 'generically continuous' for finite metric spaces?cited by 1
  9. 2025A Quotient Homology Theory of Representation in Neural Networkscited by 1
  10. 2025Stability, Complexity and Data-Dependent Worst-Case Generalization Boundscited by 1
  11. 2026A Universal Nearest-Neighbor Estimator for Intrinsic Dimensionalitycited by 1
  12. 2025Maximum diversity and weighting for invariants of periodic time seriescited by 0
3 more
  1. 2026Cardinality augmented loss functionscited by 0
  2. 2026Measuring What Persists: Conditioning Mechanisms and a Geometric Framework for AI Agent Identitycited by 0
  3. 2026Generalization in Neural Networks Through the Lens of Magnitude Potentialcited by 0