# Model-centric Data Manifold: the Data Through the Eyes of the Model

arXiv: https://arxiv.org/abs/2104.13289 · HTML: https://mlsanity.com/paper/2104.13289/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 33 references that are in the map.
- First posted on arXiv 26 Apr 2021.

## Builds on (most cited first, 16 of 16)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Conditional Image Generation with PixelCNN Decoders](https://mlsanity.com/paper/1606.05328/) (2016)
- [Towards Principled Methods for Training Generative Adversarial Networks](https://mlsanity.com/paper/1701.04862/) (2017)
- [New insights and perspectives on the natural gradient method](https://mlsanity.com/paper/1412.1193/) (2014)
- [Intrinsic dimension of data representations in deep neural networks](https://mlsanity.com/paper/1905.12784/) (2019)
- [The Hessian Penalty: A Weak Prior for Unsupervised Disentanglement](https://mlsanity.com/paper/2008.10599/) (2020)
- [An elementary introduction to information geometry](https://mlsanity.com/paper/1808.08271/) (2018)
- [Pathological spectra of the Fisher information metric and its variants in deep neural networks](https://mlsanity.com/paper/1910.05992/) (2019)
- [Towards a topological-geometrical theory of group equivariant non-expansive operators for data analysis and machine learning](https://mlsanity.com/paper/1812.11832/) (2018)
- [Horizontal Flows and Manifold Stochastics in Geometric Deep Learning](https://mlsanity.com/paper/1909.06397/) (2019)
- [Ricci Curvature and the Manifold Learning Problem](https://mlsanity.com/paper/1410.3351/) (2014)

## Built on by (most cited first, 9 of 9)

- [Elastic Feature Consolidation for Cold Start Exemplar-Free Incremental Learning](https://mlsanity.com/paper/2402.03917/) (2024)
- [Orthogonal Uncertainty Representation of Data Manifold for Robust Long-Tailed Learning](https://mlsanity.com/paper/2310.10090/) (2023)
- [Deep Learning and Geometric Deep Learning: an introduction for mathematicians and physicists](https://mlsanity.com/paper/2305.05601/) (2023)
- [EFC++: Elastic Feature Consolidation with Prototype Re-balancing for Cold Start Exemplar-free Incremental Learning](https://mlsanity.com/paper/2503.10439/) (2025)
- [Data-Centric Long-Tailed Image Recognition](https://mlsanity.com/paper/2311.01744/) (2023)
- [Geometry of Singular Foliations and Learning Manifolds in ReLU Networks via the Data Information Matrix](https://mlsanity.com/paper/2409.07412/) (2024)
- [Cartan moving frames and the data manifolds](https://mlsanity.com/paper/2409.12057/) (2024)
- [Unveiling Transformer Perception by Exploring Input Manifolds](https://mlsanity.com/paper/2410.06019/) (2024)
- [The Data Manifold under the Microscope](https://mlsanity.com/paper/2606.15760/) (2026)

## Abstract (arXiv)

We discover that deep ReLU neural network classifiers can see a low-dimensional Riemannian manifold structure on data. Such structure comes via the local data matrix, a variation of the Fisher information matrix, where the role of the model parameters is taken by the data variables. We obtain a foliation of the data domain and we show that the dataset on which the model is trained lies on a leaf, the data leaf, whose dimension is bounded by the number of classification labels. We validate our results with some experiments with the MNIST dataset: paths on the data leaf connect valid images, while other leaves cover noisy images.