# Metric Space Magnitude and Generalisation in Neural Networks

arXiv: https://arxiv.org/abs/2305.05611 · HTML: https://mlsanity.com/paper/2305.05611/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 39 references that are in the map.
- First posted on arXiv 9 May 2023.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks](https://mlsanity.com/paper/1901.06053/) (2019)
- [Traditional and Heavy-Tailed Self Regularization in Neural Network Models](https://mlsanity.com/paper/1901.08276/) (2019)
- [Topological Deep Learning: Going Beyond Graph Data](https://mlsanity.com/paper/2206.00606/) (2022)
- [A Topology Layer for Machine Learning](https://mlsanity.com/paper/1905.12200/) (2019)
- [Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topology](https://mlsanity.com/paper/1812.09764/) (2018)
- [Intrinsic Dimension, Persistent Homology and Generalization in Neural Networks](https://mlsanity.com/paper/2111.13171/) (2021)
- [Hausdorff Dimension, Heavy Tails, and Generalization in Neural Networks](https://mlsanity.com/paper/2006.09313/) (2020)
- [Persistent Homology and the Upper Box Dimension](https://mlsanity.com/paper/1802.00533/) (2018)
- [Topology and geometry of data manifold in deep learning](https://mlsanity.com/paper/2204.08624/) (2022)
- [Weighting vectors for machine learning: numerical harmonic analysis applied to boundary detection](https://mlsanity.com/paper/2106.00827/) (2021)
- [The magnitude vector of images](https://mlsanity.com/paper/2110.15188/) (2021)
- [A Primer on Persistent Homology of Finite Metric Spaces](https://mlsanity.com/paper/1905.13400/) (2019)

## Built on by (most cited first, 15 of 15)

- [Position: Topological Deep Learning is the New Frontier for Relational Learning](https://mlsanity.com/paper/2402.08871/) (2024)
- [Metric Space Magnitude for Evaluating the Diversity of Latent Representations](https://mlsanity.com/paper/2311.16054/) (2023)
- [The Effect of Intrinsic Dataset Properties on Generalization: Unraveling Learning Differences Between Natural and Medical Images](https://mlsanity.com/paper/2401.08865/) (2024)
- [Topological Data Analysis for Neural Network Analysis: A Comprehensive Survey](https://mlsanity.com/paper/2312.05840/) (2023)
- [Geometry-Aware Edge Pooling for Graph Neural Networks](https://mlsanity.com/paper/2506.11700/) (2025)
- [Approximating Metric Magnitude of Point Sets](https://mlsanity.com/paper/2409.04411/) (2024)
- [Magnitude Distance: A Geometric Measure of Dataset Similarity](https://mlsanity.com/paper/2602.08859/) (2026)
- [Is magnitude 'generically continuous' for finite metric spaces?](https://mlsanity.com/paper/2501.08745/) (2025)
- [A Quotient Homology Theory of Representation in Neural Networks](https://mlsanity.com/paper/2502.01360/) (2025)
- [Stability, Complexity and Data-Dependent Worst-Case Generalization Bounds](https://mlsanity.com/paper/2507.06775/) (2025)
- [A Universal Nearest-Neighbor Estimator for Intrinsic Dimensionality](https://mlsanity.com/paper/2603.10493/) (2026)
- [Maximum diversity and weighting for invariants of periodic time series](https://mlsanity.com/paper/2509.11146/) (2025)
- [Cardinality augmented loss functions](https://mlsanity.com/paper/2601.04941/) (2026)
- [Measuring What Persists: Conditioning Mechanisms and a Geometric Framework for AI Agent Identity](https://mlsanity.com/paper/2606.21843/) (2026)
- [Generalization in Neural Networks Through the Lens of Magnitude Potential](https://mlsanity.com/paper/2610.01633/) (2026)

## Abstract (arXiv)

Deep learning models have seen significant successes in numerous applications, but their inner workings remain elusive. The purpose of this work is to quantify the learning process of deep neural networks through the lens of a novel topological invariant called magnitude. Magnitude is an isometry invariant; its properties are an active area of research as it encodes many known invariants of a metric space. We use magnitude to study the internal representations of neural networks and propose a new method for determining their generalisation capabilities. Moreover, we theoretically connect magnitude dimension and the generalisation error, and demonstrate experimentally that the proposed framework can be a good indicator of the latter.