# Phase codes emerge in recurrent neural networks optimized for modular arithmetic

arXiv: https://arxiv.org/abs/2310.07908 · HTML: https://mlsanity.com/paper/2310.07908/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 28 references that are in the map.
- First posted on arXiv 11 Oct 2023.

## Builds on (most cited first, 5 of 5)

- [Universality and individuality in neural dynamics across large populations of recurrent networks](https://mlsanity.com/paper/1907.08549/) (2019)
- [Beyond Geometry: Comparing the Temporal Structure of Computation in Neural Circuits with Dynamical Similarity Analysis](https://mlsanity.com/paper/2306.10168/) (2023)
- [Theory of gating in recurrent neural networks](https://mlsanity.com/paper/2007.14823/) (2020)
- [Charting and navigating the space of solutions for recurrent neural networks](https://mlsanity.com/paper/2111.09356/) (2021)
- [Persistent learning signals and working memory without continuous attractors](https://mlsanity.com/paper/2308.12585/) (2023)

## Built on by (most cited first, 2 of 2)

- [Measuring and Controlling Solution Degeneracy across Task-Trained Recurrent Neural Networks](https://mlsanity.com/paper/2410.03972/) (2024)
- [Traversing the solution space of neural networks with Hessian Null Space Continuation](https://mlsanity.com/paper/2609.38081/) (2026)

## Abstract (arXiv)

Recurrent neural networks (RNNs) can implement complex computations by leveraging a range of dynamics, such as oscillations, attractors, and transient trajectories. A growing body of work has highlighted the emergence of phase codes, a type of oscillatory activity where information is encoded in the relative phase of network activity, in RNNs trained for working memory tasks. However, these studies rely on architectural constraints or regularization schemes that explicitly promote oscillatory solutions. Here, we investigate whether phase coding can emerge purely from task optimization by training continuous-time RNNs to perform a simple modular arithmetic task without oscillatory-promoting biases. We find that in the absence of such biases, RNNs can learn phase code solutions. Surprisingly, we also uncover a rich diversity of alternative solutions that solve our modular arithmetic task via qualitatively distinct dynamics and dynamical mechanisms. We map the solution space for our task and show that the phase code solution occupies a distinct region. These results suggest that phase coding can be a natural but not inevitable outcome of training RNNs on modular arithmetic, and highlight the diversity of solutions RNNs can learn to solve simple tasks.