# Space filling positionality and the Spiroformer

arXiv: https://arxiv.org/abs/2507.08456 · HTML: https://mlsanity.com/paper/2507.08456/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 22 references that are in the map.
- First posted on arXiv 11 Jul 2025.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Spherical CNNs](https://mlsanity.com/paper/1801.10130/) (2018)
- [Spherical Transformer for LiDAR-based 3D Recognition](https://mlsanity.com/paper/2303.12766/) (2023)
- [Geographic Location Encoding with Spherical Harmonics and Sinusoidal Representation Networks](https://mlsanity.com/paper/2310.06743/) (2023)
- [Symmetry Group Equivariant Architectures for Physics](https://mlsanity.com/paper/2203.06153/) (2022)
- [A Structure-Preserving Kernel Method for Learning Hamiltonian Systems](https://mlsanity.com/paper/2403.10070/) (2024)
- [Geomstats: A Python Package for Riemannian Geometry in Machine Learning](https://mlsanity.com/paper/2004.04667/) (2020)
- [Spatioformer: A Geo-encoded Transformer for Large-Scale Plant Species Richness Prediction](https://mlsanity.com/paper/2410.19256/) (2024)
- [SymFlux: deep symbolic regression of Hamiltonian vector fields](https://mlsanity.com/paper/2507.06342/) (2025)

## Abstract (arXiv)

Transformers excel when dealing with sequential data. Generalizing transformer models to geometric domains, such as manifolds, we encounter the problem of not having a well-defined global order. We propose a solution with attention heads following a space-filling curve. As a first experimental example, we present the Spiroformer, a transformer that follows a polar spiral on the 2-sphere.