BSA: Ball Sparse Attention for Large-scale Geometries
Not on the map. Why?
Abstract (arXiv)
Self-attention scales quadratically with input size, limiting its use for large-scale physical systems. Although sparse attention mechanisms provide a viable alternative, they are primarily designed for regular structures such as text or images, making them inapplicable for irregular geometries. In this work, we present Ball Sparse Attention (BSA), which adapts Native Sparse Attention (NSA) (Yuan et al., 2025) to unordered point sets by imposing regularity using the Ball Tree structure from the Erwin Transformer (Zhdanov et al., 2025). We modify NSA's components to work with ball-based neighborhoods, yielding a global receptive field at sub-quadratic cost. On an airflow pressure prediction task, we achieve accuracy comparable to Full Attention while significantly reducing the theoretical computational complexity. Our implementation is available at https://github.com/britacatalin/bsa.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 20 of its 30 references that are in the map. First posted on arXiv 14 Jun 2025.
Builds on
20 of 30 references.
References · 20
- 2017Attention Is All You Needcited by 60,361
- 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
- 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
- 2020GLU Variants Improve Transformercited by 1,804
- 2020Big Bird: Transformers for Longer Sequencescited by 1,684
- 2019Root Mean Square Layer Normalizationcited by 1,480
- 2023Point Transformer V3: Simpler, Faster, Strongercited by 589
- 2023GNOT: A General Neural Operator Transformer for Operator Learningcited by 336
- 2024Transolver: A Fast Transformer Solver for PDEs on General Geometriescited by 326
- 2023Geometry-Informed Neural Operator for Large-Scale 3D PDEscited by 303
- 2022Transformer for Partial Differential Equations' Operator Learningcited by 283
- 2023Solving High-Dimensional PDEs with Latent Spectral Modelscited by 89
8 more
- 2024Latent Neural Operator for Solving Forward and Inverse PDE Problemscited by 80
- 2023FlatFormer: Flattened Window Attention for Efficient Point Cloud Transformercited by 73
- 2023Improved Operator Learning by Orthogonal Attentioncited by 61
- 2024Universal Physics Transformers: A Framework For Efficiently Scaling Neural Operatorscited by 27
- 2025Erwin: A Tree-based Hierarchical Transformer for Large-scale Physical Systemscited by 22
- 2024Locality-Sensitive Hashing-Based Efficient Point Transformer with Applications in High-Energy Physicscited by 12
- 2023Transformers are efficient hierarchical chemical graph learnerscited by 4
- 2024Resolution-Agnostic Transformer-based Climate Downscalingcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.