# RNNs perform task computations by dynamically warping neural representations

arXiv: https://arxiv.org/abs/2512.04310 · HTML: https://mlsanity.com/paper/2512.04310/

## Key facts

- Cited by 1 papers in the map (data through 7 Oct 2026).
- Builds on 23 of its 75 references that are in the map.
- First posted on arXiv 3 Dec 2025.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task](https://mlsanity.com/paper/2210.13382/) (2022)
- [On Neural Differential Equations](https://mlsanity.com/paper/2202.02435/) (2022)
- [How to Train Your HiPPO: State Space Models with Generalized Orthogonal Basis Projections](https://mlsanity.com/paper/2206.12037/) (2022)
- [Attractor and integrator networks in the brain](https://mlsanity.com/paper/2112.03978/) (2021)
- [Neural population geometry: An approach for understanding biological and artificial neural networks](https://mlsanity.com/paper/2104.07059/) (2021)
- [Linking connectivity, dynamics and computations in low-rank recurrent neural networks](https://mlsanity.com/paper/1711.09672/) (2017)
- [Equivariant Transformer Networks](https://mlsanity.com/paper/1901.11399/) (2019)
- [Your diffusion model secretly knows the dimension of the data manifold](https://mlsanity.com/paper/2212.12611/) (2022)
- [DynGFN: Towards Bayesian Inference of Gene Regulatory Networks with GFlowNets](https://mlsanity.com/paper/2302.04178/) (2023)
- [Data-Driven Modeling and Prediction of Non-Linearizable Dynamics via Spectral Submanifolds](https://mlsanity.com/paper/2201.04976/) (2022)
- [On the Role of Fixed Points of Dynamical Systems in Training Physics-Informed Neural Networks](https://mlsanity.com/paper/2203.13648/) (2022)
- [The interplay between randomness and structure during learning in RNNs](https://mlsanity.com/paper/2006.11036/) (2020)
- [Reverse engineering recurrent neural networks with Jacobian switching linear dynamical systems](https://mlsanity.com/paper/2111.01256/) (2021)
- [Understanding deep features with computer-generated imagery](https://mlsanity.com/paper/1506.01151/) (2015)
- [Low Tensor Rank Learning of Neural Dynamics](https://mlsanity.com/paper/2308.11567/) (2023)
- [A singular Riemannian geometry approach to Deep Neural Networks I. Theoretical foundations](https://mlsanity.com/paper/2201.09656/) (2021)
- [The geometry of integration in text classification RNNs](https://mlsanity.com/paper/2010.15114/) (2020)
- [Recurrent neural network models for working memory of continuous variables: activity manifolds, connectivity patterns, and dynamic codes](https://mlsanity.com/paper/2111.01275/) (2021)
- [Emergent Riemannian geometry over learning discrete computations on continuous manifolds](https://mlsanity.com/paper/2512.00196/) (2025)
- [Discrete, compositional, and symbolic representations through attractor dynamics](https://mlsanity.com/paper/2310.01807/) (2023)

## Built on by (most cited first, 1 of 1)

- [Minimal Embodiment Enables Efficient Learning of Number Concepts in Robot](https://mlsanity.com/paper/2604.11373/) (2026)

## Abstract (arXiv)

Analysing how neural networks represent data features in their activations can help interpret how they perform tasks. Hence, a long line of work has focused on mathematically characterising the geometry of such "neural representations." In parallel, machine learning has seen a surge of interest in understanding how dynamical systems perform computations on time-varying input data. Yet, the link between computation-through-dynamics and representational geometry remains poorly understood. Here, we hypothesise that recurrent neural networks (RNNs) perform computations by dynamically warping their representations of task variables. To test this hypothesis, we develop a Riemannian geometric framework that enables the derivation of the manifold topology and geometry of a dynamical system from the manifold of its inputs. By characterising the time-varying geometry of RNNs, we show that dynamic warping is a fundamental feature of their computations.