# Transformer Heads Looking for Order

arXiv: https://arxiv.org/abs/2609.25588 · HTML: https://mlsanity.com/paper/2609.25588/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 3 of its 7 references that are in the map.
- First posted on arXiv 22 Sep 2026.

## Builds on (most cited first, 3 of 3)

- [Parity, Sensitivity, and Transformers](https://mlsanity.com/paper/2602.05896/) (2026)
- [The Head Complexity of Boolean Functions in Single-Layer Attention](https://mlsanity.com/paper/2609.04046/) (2026)
- [The relaxation complexity of the standard simplex is logarithmic](https://mlsanity.com/paper/2606.11852/) (2026)

## Abstract (arXiv)

In this note, we show that the problem of checking, whether a sequence of bits is ordered, is not doable by 1-head 1-layer transformers but is doable by a 2-head 1-layer transformer. Unlike similar previous results, our results assume the model where transformers have an output MLP.