# The Data Manifold under the Microscope

arXiv: https://arxiv.org/abs/2606.15760 · HTML: https://mlsanity.com/paper/2606.15760/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 74 references that are in the map.
- First posted on arXiv 14 Jun 2026.

## Builds on (most cited first, 29 of 29)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Measuring the Intrinsic Dimension of Objective Landscapes](https://mlsanity.com/paper/1804.08838/) (2018)
- [The Intrinsic Dimension of Images and Its Impact on Learning](https://mlsanity.com/paper/2104.08894/) (2021)
- [Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality](https://mlsanity.com/paper/1801.02613/) (2018)
- [Intrinsic dimension of data representations in deep neural networks](https://mlsanity.com/paper/1905.12784/) (2019)
- [Latent Space Oddity: on the Curvature of Deep Generative Models](https://mlsanity.com/paper/1710.11379/) (2017)
- [Intrinsic Dimension, Persistent Homology and Generalization in Neural Networks](https://mlsanity.com/paper/2111.13171/) (2021)
- [Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connections](https://mlsanity.com/paper/2404.02954/) (2024)
- [Chart Auto-Encoders for Manifold Structured Data](https://mlsanity.com/paper/1912.10094/) (2019)
- [Do Not Escape From the Manifold: Discovering the Local Coordinates on the Latent Space of GANs](https://mlsanity.com/paper/2106.06959/) (2021)
- [Hardness of Learning Neural Networks under the Manifold Hypothesis](https://mlsanity.com/paper/2406.01461/) (2024)
- [Efficient Representation of Low-Dimensional Manifolds using Deep Networks](https://mlsanity.com/paper/1602.04723/) (2016)
- [Manifold Approximation by Moving Least-Squares Projection (MMLS)](https://mlsanity.com/paper/1606.07104/) (2016)
- [Efficient Manifold and Subspace Approximations with Spherelets](https://mlsanity.com/paper/1706.08263/) (2017)
- [Data Representations' Study of Latent Image Manifolds](https://mlsanity.com/paper/2305.19730/) (2023)
- [Max-Affine Spline Insights into Deep Generative Networks](https://mlsanity.com/paper/2002.11912/) (2020)
- [Manifold Fitting under Unbounded Noise](https://mlsanity.com/paper/1909.10228/) (2019)
- [Model-centric Data Manifold: the Data Through the Eyes of the Model](https://mlsanity.com/paper/2104.13289/) (2021)
- [Manifold Learning Using Kernel Density Estimation and Local Principal Components Analysis](https://mlsanity.com/paper/1709.03615/) (2017)
- [Estimating the reach of a manifold via its convexity defect function](https://mlsanity.com/paper/2001.08006/) (2020)
- [Quantifying Extrinsic Curvature in Neural Manifolds](https://mlsanity.com/paper/2212.10414/) (2022)
- [Ricci Curvature and the Manifold Learning Problem](https://mlsanity.com/paper/1410.3351/) (2014)
- [A Theoretical Study of Neural Network Expressive Power via Manifold Topology](https://mlsanity.com/paper/2410.16542/) (2024)
- [A Multi-criteria Approach for Fast and Outlier-aware Representative Selection from Manifolds](https://mlsanity.com/paper/2003.05989/) (2020)
- [Blessing of Dimensionality for Approximating Sobolev Classes on Manifolds](https://mlsanity.com/paper/2408.06996/) (2024)

## Abstract (arXiv)

A significant gap exists between theory and practice in deep learning. Generalization and approximation error bounds are often derived for simplified models or are too loose to be informative. Many rely on the manifold hypothesis and on geometric regularity such as intrinsic dimension, curvature, and reach. Progress requires insight into data-manifold geometry and suitable benchmarks, yet existing options are polarized: analytic manifolds with known geometry but limited applicability, or real-world datasets where geometry is only coarsely estimable. We introduce a benchmarking framework for studying data geometry. We repurpose and extend dSprites and COIL-20 with additional transformation dimensions and dense, axis-aligned sampling, and pair them with finite-difference estimators that recover curvature, reach, and volume at near-ground-truth accuracy in a regime where general-purpose estimators are unreliable or difficult to deploy. The framework is intended as a controlled testbed, useful as a calibration environment for geometric estimators and a sandbox for probing theoretical assumptions. To illustrate its use, we present two application studies, namely assessing the scaling behavior of the bounds of Genovese et al. and Fefferman et al., and tracking the layer-wise geometry of a β-VAE, highlighting the behavior of current bounds and the value of controlled benchmarks for guiding and validating future theory. A reference implementation is available at https://github.com/koulakis/manifold-microscope.