Space filling positionality and the Spiroformer
Not on the map. Why?
Abstract (arXiv)
Transformers excel when dealing with sequential data. Generalizing transformer models to geometric domains, such as manifolds, we encounter the problem of not having a well-defined global order. We propose a solution with attention heads following a space-filling curve. As a first experimental example, we present the Spiroformer, a transformer that follows a polar spiral on the 2-sphere.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 10 of its 22 references that are in the map. First posted on arXiv 11 Jul 2025.
Builds on
10 of 22 references.
References · 10
- 2017Attention Is All You Needcited by 60,361
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2018Spherical CNNscited by 456
- 2023Spherical Transformer for LiDAR-based 3D Recognitioncited by 125
- 2023Geographic Location Encoding with Spherical Harmonics and Sinusoidal Representation Networkscited by 61
- 2022Symmetry Group Equivariant Architectures for Physicscited by 23
- 2024A Structure-Preserving Kernel Method for Learning Hamiltonian Systemscited by 9
- 2020Geomstats: A Python Package for Riemannian Geometry in Machine Learningcited by 6
- 2024Spatioformer: A Geo-encoded Transformer for Large-Scale Plant Species Richness Predictioncited by 4
- 2025SymFlux: deep symbolic regression of Hamiltonian vector fieldscited by 1
Built on by
Not cited yet.