mlsanityData to 7 Oct 2026

Model-centric Data Manifold: the Data Through the Eyes of the Model

arXiv 2104.13289 ↗First posted 26 Apr 2021cs.LG math.OC stat.MLcited by 9

Not on the map. Why?

Abstract (arXiv)

We discover that deep ReLU neural network classifiers can see a low-dimensional Riemannian manifold structure on data. Such structure comes via the local data matrix, a variation of the Fisher information matrix, where the role of the model parameters is taken by the data variables. We obtain a foliation of the data domain and we show that the dataset on which the model is trained lies on a leaf, the data leaf, whose dimension is bounded by the number of classification labels. We validate our results with some experiments with the MNIST dataset: paths on the data leaf connect valid images, while other leaves cover noisy images.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 16 of its 33 references that are in the map. First posted on arXiv 26 Apr 2021.

Builds on

16 of 33 references.

References · 16

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  5. 2013Auto-Encoding Variational Bayescited by 3,980
  6. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  7. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  8. 2017Towards Principled Methods for Training Generative Adversarial Networkscited by 460
  9. 2014New insights and perspectives on the natural gradient methodcited by 435
  10. 2019Intrinsic dimension of data representations in deep neural networkscited by 276
  11. 2020The Hessian Penalty: A Weak Prior for Unsupervised Disentanglementcited by 90
  12. 2018An elementary introduction to information geometrycited by 76
4 more
  1. 2019Pathological spectra of the Fisher information metric and its variants in deep neural networkscited by 32
  2. 2018Towards a topological-geometrical theory of group equivariant non-expansive operators for data analysis and machine learningcited by 24
  3. 2019Horizontal Flows and Manifold Stochastics in Geometric Deep Learningcited by 6
  4. 2014Ricci Curvature and the Manifold Learning Problemcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 4; 2025: 1; 2026: 1.

9 papers.

  1. 2024Elastic Feature Consolidation for Cold Start Exemplar-Free Incremental Learningcited by 37
  2. 2023Orthogonal Uncertainty Representation of Data Manifold for Robust Long-Tailed Learningcited by 5
  3. 2023Deep Learning and Geometric Deep Learning: an introduction for mathematicians and physicistscited by 3
  4. 2025EFC++: Elastic Feature Consolidation with Prototype Re-balancing for Cold Start Exemplar-free Incremental Learningcited by 2
  5. 2023Data-Centric Long-Tailed Image Recognitioncited by 1
  6. 2024Geometry of Singular Foliations and Learning Manifolds in ReLU Networks via the Data Information Matrixcited by 1
  7. 2024Cartan moving frames and the data manifoldscited by 1
  8. 2024Unveiling Transformer Perception by Exploring Input Manifoldscited by 0
  9. 2026The Data Manifold under the Microscopecited by 0