# The Shape of Data: Intrinsic Distance for Data Distributions

arXiv: https://arxiv.org/abs/1905.11141 · HTML: https://mlsanity.com/paper/1905.11141/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 May 2019.

## Built on by (most cited first, 30 of 38)

- [Relative representations enable zero-shot latent space communication](https://mlsanity.com/paper/2209.15430/) (2022)
- [Similarity of Neural Network Models: A Survey of Functional and Representational Measures](https://mlsanity.com/paper/2305.06329/) (2023)
- [Pros and Cons of GAN Evaluation Measures: New Developments](https://mlsanity.com/paper/2103.09396/) (2021)
- [Unleashing Transformers: Parallel Token Prediction with Discrete Absorbing Diffusion for Fast High-Resolution Image Generation from Vector-Quantized Codes](https://mlsanity.com/paper/2111.12701/) (2021)
- [Representation Topology Divergence: A Method for Comparing Neural Network Representations](https://mlsanity.com/paper/2201.00058/) (2021)
- [Latent Space Translation via Semantic Alignment](https://mlsanity.com/paper/2311.00664/) (2023)
- [Do Vision and Language Encoders Represent the World Similarly?](https://mlsanity.com/paper/2401.05224/) (2024)
- [Evaluation Metrics for Graph Generative Models: Problems, Pitfalls, and Practical Solutions](https://mlsanity.com/paper/2106.01098/) (2021)
- [PyTorch Image Quality: Metrics for Image Quality Assessment](https://mlsanity.com/paper/2208.14818/) (2022)
- [Manifold Topology Divergence: a Framework for Comparing Data Manifolds](https://mlsanity.com/paper/2106.04024/) (2021)
- [ReSi: A Comprehensive Benchmark for Representational Similarity Measures](https://mlsanity.com/paper/2408.00531/) (2024)
- [SuperLoRA: Parameter-Efficient Unified Adaptation of Multi-Layer Attention Modules](https://mlsanity.com/paper/2403.11887/) (2024)
- [Latent Space Translation via Inverse Relative Projection](https://mlsanity.com/paper/2406.15057/) (2024)
- [Neural Lineage](https://mlsanity.com/paper/2406.11129/) (2024)
- [Bootstrapping Parallel Anchors for Relative Representations](https://mlsanity.com/paper/2303.00721/) (2023)
- [TexTile: A Differentiable Metric for Texture Tileability](https://mlsanity.com/paper/2403.12961/) (2024)
- [Neural Operator Variational Inference based on Regularized Stein Discrepancy for Deep Gaussian Processes](https://mlsanity.com/paper/2309.12658/) (2023)
- [Log-Euclidean Signatures for Intrinsic Distances Between Unaligned Datasets](https://mlsanity.com/paper/2202.01671/) (2022)
- [Mapping the Multiverse of Latent Representations](https://mlsanity.com/paper/2402.01514/) (2024)
- [Approximating Latent Manifolds in Neural Networks via Vanishing Ideals](https://mlsanity.com/paper/2502.15051/) (2025)
- [RTD-Lite: Scalable Topological Analysis for Comparing Weighted Graphs in Learning Tasks](https://mlsanity.com/paper/2503.11910/) (2025)
- [Connecting Neural Models Latent Geometries with Relative Geodesic Representations](https://mlsanity.com/paper/2506.01599/) (2025)
- [Neural FIM for learning Fisher Information Metrics from point cloud data](https://mlsanity.com/paper/2306.06062/) (2023)
- [Understanding Deep Generative Models with Generalized Empirical Likelihoods](https://mlsanity.com/paper/2306.09780/) (2023)
- [Metrics to Quantify Global Consistency in Synthetic Medical Images](https://mlsanity.com/paper/2308.00402/) (2023)
- [Mitigating the Contractivity Trap in Diffusion ODEs via Stein Stabilization](https://mlsanity.com/paper/2606.07835/) (2026)
- [A Method for Evaluating Deep Generative Models of Images via Assessing the Reproduction of High-order Spatial Context](https://mlsanity.com/paper/2111.12577/) (2021)
- [Image Quality Assessment for Magnetic Resonance Imaging](https://mlsanity.com/paper/2203.07809/) (2022)
- [FedAnchor: Enhancing Federated Semi-Supervised Learning with Label Contrastive Loss for Unlabeled Clients](https://mlsanity.com/paper/2402.10191/) (2024)
- [Manifold Approximation leads to Robust Kernel Alignment](https://mlsanity.com/paper/2510.22953/) (2025)

## Abstract (arXiv)

The ability to represent and compare machine learning models is crucial in order to quantify subtle model changes, evaluate generative models, and gather insights on neural network architectures. Existing techniques for comparing data distributions focus on global data properties such as mean and covariance; in that sense, they are extrinsic and uni-scale. We develop a first-of-its-kind intrinsic and multi-scale method for characterizing and comparing data manifolds, using a lower-bound of the spectral variant of the Gromov-Wasserstein inter-manifold distance, which compares all data moments. In a thorough experimental study, we demonstrate that our method effectively discerns the structure of data manifolds even on unaligned data of different dimensionalities; moreover, we showcase its efficacy in evaluating the quality of generative models.