# The Ball and the Box: Two Geometries of Computation in Superposition

arXiv: https://arxiv.org/abs/2610.11744 · HTML: https://mlsanity.com/paper/2610.11744/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability.
- Builds on 7 of its 45 references that are in the map.
- First posted on arXiv 8 Oct 2026.

## Placed in

- Mechanistic interpretability / [Circuits and mechanistic analysis](https://mlsanity.com/mechinterp/circuits-and-mechanistic-analysis/)

## Builds on (most cited first, 7 of 7)

- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [Mathematical Models of Computation in Superposition](https://mlsanity.com/paper/2408.05451/) (2024)
- [On the Complexity of Neural Computation in Superposition](https://mlsanity.com/paper/2409.15318/) (2024)
- [Compressed Computation is (probably) not Computation in Superposition](https://mlsanity.com/paper/2606.14673/) (2026)
- [Towards a mathematical theory of superposition](https://mlsanity.com/paper/2608.27540/) (2026)
- [Compressed Computation: Dense Circuits in a Toy Model of the Universal-AND Problem](https://mlsanity.com/paper/2507.09816/) (2025)
- [Feature Superposition in Neural Networks: From Theory to Practice](https://mlsanity.com/paper/2609.06862/) (2026)

## Abstract (arXiv)

Neural representations can encode more features than they have dimensions, a phenomenon known as superposition. We study the dimension needed to compute Boolean gates from such representations. For a single threshold layer with a Gaussian random dictionary and uniformly random sparse Boolean inputs, we derive sharp dimension thresholds under two error criteria. A vanishing expected error count can require more dimensions than correctness of every output with high probability. Shared reads explain the gap: rare realizations can produce many errors at once. The expected-count threshold has ball geometry, while joint reliability has box geometry when a gate is evaluated on every feature tuple. Optimizing shared readout weights and biases gives explicit thresholds for conjunction, disjunction, and majority. For pairwise conjunction, the analysis also describes the transition near the threshold, in agreement with exact simulations.