# Uncovering a Universal Abstract Algorithm for Modular Addition in Neural Networks

arXiv: https://arxiv.org/abs/2505.18266 · HTML: https://mlsanity.com/paper/2505.18266/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 May 2025.

## Built on by (most cited first, 10 of 10)

- [Theory of Scaling Laws for In-Context Regression: Depth, Width, Context and Time](https://mlsanity.com/paper/2510.01098/) (2025)
- [Sequential Group Composition: A Window into the Mechanics of Deep Learning](https://mlsanity.com/paper/2602.03655/) (2026)
- [Pretrain-Test Task Alignment Governs Generalization in In-Context Learning](https://mlsanity.com/paper/2509.26551/) (2025)
- [On the geometry and topology of representations: the manifolds of modular addition](https://mlsanity.com/paper/2512.25060/) (2025)
- [Can Neural Networks Learn Small Algebraic Worlds? An Investigation Into the Group-theoretic Structures Learned By Narrow Models Trained To Predict Group Operations](https://mlsanity.com/paper/2601.21150/) (2026)
- [Correspondence Between Ising Machines and Neural Networks](https://mlsanity.com/paper/2511.00746/) (2025)
- [Gradient Flow Through Diagram Expansions: Learning Regimes and Explicit Solutions](https://mlsanity.com/paper/2602.04548/) (2026)
- [Unveiling Memorization-Generalization Coexistence: A Case Study on Arithmetic Tasks with Label Noise](https://mlsanity.com/paper/2605.18022/) (2026)
- [Learning Large-Scale Modular Addition with an Auxiliary Modulus](https://mlsanity.com/paper/2605.07648/) (2026)
- [The Discrete-Log Clock: How a Transformer Learns Modular Multiplication](https://mlsanity.com/paper/2606.17399/) (2026)

## Abstract (arXiv)

We propose a testable universality hypothesis, asserting that seemingly disparate neural network solutions observed in the simple task of modular addition are unified under a common abstract algorithm. While prior work interpreted variations in neuron-level representations as evidence for distinct algorithms, we demonstrate - through multi-level analyses spanning neurons, neuron clusters, and entire networks - that multilayer perceptrons and transformers universally implement the abstract algorithm we call the approximate Chinese Remainder Theorem. Crucially, we introduce approximate cosets and show that neurons activate exclusively on them. Furthermore, our theory works for deep neural networks (DNNs). It predicts that universally learned solutions in DNNs with trainable embeddings or more than one hidden layer require only O(log n) features, a result we empirically confirm. This work thus provides the first theory-backed interpretation of multilayer networks solving modular addition. It advances generalizable interpretability and opens a testable universality hypothesis for group multiplication beyond modular addition.