# Emergence of Grid-like Representations by Training Recurrent Networks with Conformal Normalization

arXiv: https://arxiv.org/abs/2310.19192 · HTML: https://mlsanity.com/paper/2310.19192/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 70 references that are in the map.
- First posted on arXiv 29 Oct 2023.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models](https://mlsanity.com/paper/1702.03275/) (2017)
- [Relating transformers to models and neural representations of the hippocampal formation](https://mlsanity.com/paper/2112.04035/) (2021)
- [Emergence of grid-like representations by training recurrent neural networks to perform spatial localization](https://mlsanity.com/paper/1803.07770/) (2018)
- [Learning Grid Cells as Vector Representation of Self-Position Coupled with Matrix Representation of Self-Motion](https://mlsanity.com/paper/1810.05597/) (2018)
- [The Sense of Place: Grid Cells in the Brain and the Transcendental Number e](https://mlsanity.com/paper/1304.0031/) (2013)
- [Actionable Neural Representations: Grid Cells from Minimal Constraints](https://mlsanity.com/paper/2209.15563/) (2022)
- [On Path Integration of Grid Cells: Group Representation and Isotropic Scaling](https://mlsanity.com/paper/2006.10259/) (2020)
- [Conformal Isometry of Lie Group Representation in Recurrent Network of Grid Cells](https://mlsanity.com/paper/2210.02684/) (2022)
- [Probable nature of higher-dimensional symmetries underlying mammalian grid-cell activity patterns](https://mlsanity.com/paper/1411.2136/) (2014)

## Built on by (most cited first, 2 of 2)

- [A minimalistic representation model for head direction system](https://mlsanity.com/paper/2411.10596/) (2024)
- [Place Cells as Multi-Scale Position Embeddings: Random Walk Transition Kernels for Path Planning](https://mlsanity.com/paper/2505.14806/) (2025)

## Abstract (arXiv)

Grid cells in the entorhinal cortex of mammalian brains exhibit striking hexagon grid firing patterns in their response maps as the animal (e.g., a rat) navigates in a 2D open environment. In this paper, we study the emergence of the hexagon grid patterns of grid cells based on a general recurrent neural network (RNN) model that captures the navigation process. The responses of grid cells collectively form a high dimensional vector, representing the 2D self-position of the agent. As the agent moves, the vector is transformed by an RNN that takes the velocity of the agent as input. We propose a simple yet general conformal normalization of the input velocity of the RNN, so that the local displacement of the position vector in the high-dimensional neural space is proportional to the local displacement of the agent in the 2D physical space, regardless of the direction of the input velocity. We apply this mechanism to both a linear RNN and nonlinear RNNs. Theoretically, we provide an understanding that explains the connection between conformal normalization and the emergence of hexagon grid patterns. Empirically, we conduct extensive experiments to verify that conformal normalization is crucial for the emergence of hexagon grid patterns, across various types of RNNs. The learned patterns share similar profiles to biological grid cells, and the topological properties of the patterns also align with our theoretical understanding.