# Partial Answer of How Transformers Learn Automata

arXiv: https://arxiv.org/abs/2504.20395 · HTML: https://mlsanity.com/paper/2504.20395/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 2 of its 6 references that are in the map.
- First posted on arXiv 29 Apr 2025.

## Builds on (most cited first, 2 of 2)

- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Transformers Learn Shortcuts to Automata](https://mlsanity.com/paper/2210.10749/) (2022)

## Abstract (arXiv)

We introduce a novel framework for simulating finite automata using representation-theoretic semidirect products and Fourier modules, achieving more efficient Transformer-based implementations.