# Emergent Riemannian geometry over learning discrete computations on continuous manifolds

arXiv: https://arxiv.org/abs/2512.00196 · HTML: https://mlsanity.com/paper/2512.00196/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 26 references that are in the map.
- First posted on arXiv 28 Nov 2025.

## Builds on (most cited first, 18 of 18)

- [Exact solutions to the nonlinear dynamics of learning in deep linear neural networks](https://mlsanity.com/paper/1312.6120/) (2013)
- [Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task](https://mlsanity.com/paper/2210.13382/) (2022)
- [SGD learning on neural networks: leap complexity and saddle-to-saddle dynamics](https://mlsanity.com/paper/2302.11055/) (2023)
- [Learning curves of generic features maps for realistic datasets with a teacher-student model](https://mlsanity.com/paper/2102.08127/) (2021)
- [Understanding the Latent Space of Diffusion Models through the Lens of Riemannian Geometry](https://mlsanity.com/paper/2307.12868/) (2023)
- [Dynamics of stochastic gradient descent for two-layer neural networks in the teacher-student setup](https://mlsanity.com/paper/1906.08632/) (2019)
- [How Two-Layer Neural Networks Learn, One (Giant) Step at a Time](https://mlsanity.com/paper/2305.18270/) (2023)
- [Neural Networks Efficiently Learn Low-Dimensional Representations with SGD](https://mlsanity.com/paper/2209.14863/) (2022)
- [Neural population geometry: An approach for understanding biological and artificial neural networks](https://mlsanity.com/paper/2104.07059/) (2021)
- [Computational-Statistical Gaps in Gaussian Single-Index Models](https://mlsanity.com/paper/2403.05529/) (2024)
- [Equivariant Transformer Networks](https://mlsanity.com/paper/1901.11399/) (2019)
- [Geometric compression of invariant manifolds in neural nets](https://mlsanity.com/paper/2007.11471/) (2020)
- [Riemannian Residual Neural Networks](https://mlsanity.com/paper/2310.10013/) (2023)
- [Understanding deep features with computer-generated imagery](https://mlsanity.com/paper/1506.01151/) (2015)
- [Low Tensor Rank Learning of Neural Dynamics](https://mlsanity.com/paper/2308.11567/) (2023)
- [A singular Riemannian geometry approach to Deep Neural Networks I. Theoretical foundations](https://mlsanity.com/paper/2201.09656/) (2021)
- [Uncertainty Estimation Using Riemannian Model Dynamics for Offline Reinforcement Learning](https://mlsanity.com/paper/2102.11327/) (2021)
- [Learning single-index models via harmonic decomposition](https://mlsanity.com/paper/2506.09887/) (2025)

## Built on by (most cited first, 3 of 3)

- [Geometry-aware similarity metrics for neural representations on Riemannian and statistical manifolds](https://mlsanity.com/paper/2603.28764/) (2026)
- [RNNs perform task computations by dynamically warping neural representations](https://mlsanity.com/paper/2512.04310/) (2025)
- [Riemannian Geometry for Pre-trained Language Model Embeddings](https://mlsanity.com/paper/2607.07047/) (2026)

## Abstract (arXiv)

Many tasks require mapping continuous input data (e.g. images) to discrete task outputs (e.g. class labels). Yet, how neural networks learn to perform such discrete computations on continuous data manifolds remains poorly understood. Here, we show that signatures of such computations emerge in the representational geometry of neural networks as they learn. By analysing the Riemannian pullback metric across layers of a neural network, we find that network computation can be decomposed into two functions: discretising continuous input features and performing logical operations on these discretised variables. Furthermore, we demonstrate how different learning regimes (rich vs. lazy) have contrasting metric and curvature structures, affecting the ability of the networks to generalise to unseen inputs. Overall, our work provides a geometric framework for understanding how neural networks learn to perform discrete computations on continuous manifolds.