# A singular Riemannian geometry approach to Deep Neural Networks I. Theoretical foundations

arXiv: https://arxiv.org/abs/2201.09656 · HTML: https://mlsanity.com/paper/2201.09656/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 51 references that are in the map.
- First posted on arXiv 17 Dec 2021.

## Builds on (most cited first, 16 of 16)

- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [Geometric deep learning on graphs and manifolds using mixture model CNNs](https://mlsanity.com/paper/1611.08402/) (2016)
- [Deep Convolutional Networks on Graph-Structured Data](https://mlsanity.com/paper/1506.05163/) (2015)
- [Universal Approximation with Deep Narrow Networks](https://mlsanity.com/paper/1905.08539/) (2019)
- [An overview of deep learning in medical imaging focusing on MRI](https://mlsanity.com/paper/1811.10052/) (2018)
- [Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activations](https://mlsanity.com/paper/1708.02691/) (2017)
- [Learning shape correspondence with anisotropic convolutional neural networks](https://mlsanity.com/paper/1605.06437/) (2016)
- [A Survey of Deep Learning Techniques for Neural Machine Translation](https://mlsanity.com/paper/2002.07526/) (2020)
- [Non-Euclidean Universal Approximation](https://mlsanity.com/paper/2006.02341/) (2020)
- [A survey on Deep Learning Advances on Different 3D Data Representations](https://mlsanity.com/paper/1808.01462/) (2018)
- [Community detection in node-attributed social networks: a survey](https://mlsanity.com/paper/1912.09816/) (2019)
- [Non-linear dimensionality reduction: Riemannian metric estimation and the problem of geometric discovery](https://mlsanity.com/paper/1305.7255/) (2013)
- [Deep Neural Networks for Automatic Speech Processing: A Survey from Large Corpora to Limited Data](https://mlsanity.com/paper/2003.04241/) (2020)
- [Multi-view Data Visualisation via Manifold Learning](https://mlsanity.com/paper/2101.06763/) (2021)
- [Rehabilitating Isomap: Euclidean Representation of Geodesic Structure](https://mlsanity.com/paper/2006.10858/) (2020)
- [A Differential Topological View of Challenges in Learning with Feedforward Neural Networks](https://mlsanity.com/paper/1811.10304/) (2018)

## Built on by (most cited first, 11 of 11)

- [Origami in N dimensions: How feed-forward networks manufacture linear separability](https://mlsanity.com/paper/2203.11355/) (2022)
- [Deep Learning as Ricci Flow](https://mlsanity.com/paper/2404.14265/) (2024)
- [Probabilistic Risk Assessment of an Obstacle Detection System for GoA 4 Freight Trains](https://mlsanity.com/paper/2306.14814/) (2023)
- [Emergent Riemannian geometry over learning discrete computations on continuous manifolds](https://mlsanity.com/paper/2512.00196/) (2025)
- [A singular Riemannian Geometry Approach to Deep Neural Networks III. Piecewise Differentiable Layers and Random Walks on n-dimensional Classes](https://mlsanity.com/paper/2404.06104/) (2024)
- [GeloVec: Higher Dimensional Geometric Smoothing for Coherent Visual Feature Extraction in Image Segmentation](https://mlsanity.com/paper/2505.01057/) (2025)
- [RNNs perform task computations by dynamically warping neural representations](https://mlsanity.com/paper/2512.04310/) (2025)
- [Unveiling Transformer Perception by Exploring Input Manifolds](https://mlsanity.com/paper/2410.06019/) (2024)
- [Prime Convolutional Model: Breaking the Ground for Theoretical Explainability](https://mlsanity.com/paper/2503.02773/) (2025)
- [Dual-Manifold Geometry Guided Representation Learning: Adaptive Coupling between Kernel and Data Spaces](https://mlsanity.com/paper/2608.12737/) (2026)
- [Task-Induced Riemannian Metrics for Vision Transformer Feature Spaces](https://mlsanity.com/paper/2609.27988/) (2026)

## Abstract (arXiv)

Deep Neural Networks are widely used for solving complex problems in several scientific areas, such as speech recognition, machine translation, image analysis. The strategies employed to investigate their theoretical properties mainly rely on Euclidean geometry, but in the last years new approaches based on Riemannian geometry have been developed. Motivated by some open problems, we study a particular sequence of maps between manifolds, with the last manifold of the sequence equipped with a Riemannian metric. We investigate the structures induced trough pullbacks on the other manifolds of the sequence and on some related quotients. In particular, we show that the pullbacks of the final Riemannian metric to any manifolds of the sequence is a degenerate Riemannian metric inducing a structure of pseudometric space, we show that the Kolmogorov quotient of this pseudometric space yields a smooth manifold, which is the base space of a particular vertical bundle. We investigate the theoretical properties of the maps of such sequence, eventually we focus on the case of maps between manifolds implementing neural networks of practical interest and we present some applications of the geometric framework we introduced in the first part of the paper.