mlsanityData to 14 Oct 2026

Geometric Algebra Transformer

arXiv 2305.18415 ↗First posted 28 May 2023cs.LG cs.RO stat.MLcited by 48

Not on the map. Why?

Abstract (arXiv)

Problems involving geometric data arise in physics, chemistry, robotics, computer vision, and many other fields. Such data can take numerous forms, for instance points, direction vectors, translations, or rotations, but to date there is no single architecture that can be applied to such a wide variety of geometric types while respecting their symmetries. In this paper we introduce the Geometric Algebra Transformer (GATr), a general-purpose architecture for geometric data. GATr represents inputs, outputs, and hidden states in the projective geometric (or Clifford) algebra, which offers an efficient 16-dimensional vector-space representation of common geometric objects as well as operators acting on them. GATr is equivariant with respect to E(3), the symmetry group of 3D Euclidean space. As a Transformer, GATr is versatile, efficient, and scalable. We demonstrate GATr in problems from n-body modeling to wall-shear-stress estimation on large arterial meshes to robotic motion planning. GATr consistently outperforms both non-geometric and equivariant baselines in terms of error, data efficiency, and scalability.

Key facts. Cited by 48 papers in the map (data through 14 Oct 2026). Builds on 43 of its 55 references that are in the map. First posted on arXiv 28 May 2023.

Builds on

43 of 55 references.

References · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  5. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  6. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  7. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  8. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  9. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  10. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  11. 2016Group Equivariant Convolutional Networkscited by 1,165
  12. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
31 more
  1. 2021E(n) Equivariant Graph Neural Networkscited by 965
  2. 2020On Layer Normalization in the Transformer Architecturecited by 922
  3. 2016Training Deep Nets with Sublinear Memory Costcited by 848
  4. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 746
  5. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
  6. 2021E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentialscited by 708
  7. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 607
  8. 2022MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fieldscited by 474
  9. 2019Axial Attention in Multidimensional Transformerscited by 358
  10. 2022Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphscited by 264
  11. 2019Cormorant: Covariant Molecular Neural Networkscited by 263
  12. 2020Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densitiescited by 250
  13. 2021Geometric and Physical Quantities Improve E(3) Equivariant Message Passingcited by 228
  14. 2018Adaptive Input Representations for Neural Language Modelingcited by 226
  15. 2021Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulationcited by 209
  16. 2021Self-attention Does Not Need O(n2) Memorycited by 154
  17. 2020Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphscited by 92
  18. 2022Clifford Neural Layers for PDE Modelingcited by 92
  19. 2020Attentive Group Equivariant Convolutional Networkscited by 63
  20. 2021Equivariant Q Learning in Spatial Action Spacescited by 62
  21. 2023Clifford Group Equivariant Neural Networkscited by 55
  22. 2022Sample Efficient Grasp Learning Using Equivariant Modelscited by 53
  23. 2022Equivariant Transporter Networkcited by 46
  24. 2022On-Robot Learning With Equivariant Modelscited by 42
  25. 2022So3krates: Equivariant attention for interactions on arbitrary length-scales in molecular systemscited by 39
  26. 2023Geometric Clifford Algebra Networkscited by 38
  27. 2023EDGI: Equivariant Diffusion for Planning with Embodied Agentscited by 36
  28. 2020Roto-Translation Equivariant Convolutional Networks: Application to Histopathology Image Analysiscited by 32
  29. 2022Mesh Neural Networks for SE(3)-Equivariant Hemodynamics Estimation on the Artery Wallcited by 29
  30. 2021Graded Symmetry Groups: Plane and Simplecited by 11
  31. 2021Geometric Algebra Attention Networks for Small Point Cloudscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 17; 2025: 13; 2026: 17.

48 papers.

Sort:

By area:

  1. 2024Iterated Denoising Energy Matching for Sampling from Boltzmann Densitiescited by 116
  2. 2023A Hitchhiker's Guide to Geometric GNNs for 3D Atomic Systemscited by 68
  3. 2024Does equivariance matter at scale?cited by 52
  4. 2024A Lorentz-Equivariant Transformer for All of the LHCcited by 28
  5. 2024Clifford-Steerable Convolutional Neural Networkscited by 25
  6. 2024E(n) Equivariant Topological Neural Networkscited by 22
  7. 2025Erwin: A Tree-based Hierarchical Transformer for Large-scale Physical Systemscited by 22
  8. 2024Beyond Euclid: An Illustrated Guide to Modern Machine Learning with Geometric, Topological, and Algebraic Structurescited by 20
  9. 2024Relaxed Equivariance via Multitask Learningcited by 13
  10. 2024LaB-GATr: geometric algebra transformers for large biomedical surface and volume meshescited by 12
  11. 2025Flopping for FLOPs: Leveraging equivariance for computational efficiencycited by 8
  12. 2025GAGrasp: Geometric Algebra Diffusion for Dexterous Graspingcited by 8
36 more
  1. 2026CliffordNet: All You Need is Geometric Algebracited by 7
  2. 2024Differentiable and Learnable Wireless Simulation with Geometric Transformerscited by 6
  3. 2024SE(3)-Hyena Operator for Scalable Equivariant Learningcited by 6
  4. 2024Multivector Neurons: Better and Faster O(n)-Equivariant Clifford Graph Neural Networkscited by 5
  5. 2024Optimal Equivariant Architectures from the Symmetries of Matrix-Element Likelihoodscited by 5
  6. 2024A Cosmic-Scale Benchmark for Symmetry-Preserving Data Processingcited by 5
  7. 2025Geometric Hyena Networks for Large-scale Equivariant Learningcited by 4
  8. 2025Equivariant Neural Networks for General Linear Symmetries on Lie Algebrascited by 4
  9. 2025Clifford Group Equivariant Diffusion Models for 3D Molecular Generationcited by 3
  10. 2025Hybrid Diffusion Policies with Projective Geometric Algebra for Efficient Robot Manipulation Learningcited by 3
  11. 2025Learning Hemodynamic Scalar Fields on Coronary Artery Meshes: A Benchmark of Geometric Deep Learning Modelscited by 2
  12. 2025GLGENN: A Novel Parameter-Light Equivariant Neural Networks Architecture Based on Clifford Geometric Algebrascited by 2
  13. 2026Versor: A Geometric Sequence Architecturecited by 2
  14. 2026End-to-end event reconstruction for precision physics at future colliderscited by 2
  15. 2024Randomized Geometric Algebra Methods for Convex Neural Networkscited by 1
  16. 2024Geometric Algebra Meets Large Language Models: Instruction-Based Transformations of Separate Meshes in 3D, Interactive and Controllable Scenescited by 1
  17. 2024Optimal Symmetries in Binary Classificationcited by 1
  18. 2026Efficient Equivariant Transformer for Self-Driving Agent Modelingcited by 1
  19. 2026Geometric Algebra Meets Cartesian Tensors: Higher-Order Equivariance for Interatomic Potentialscited by 1
  20. 2024STAResNet: a Network in Spacetime Algebra to solve Maxwell's PDEscited by 0
  21. 2025FoGE: Fock Space inspired encoding for graph promptingcited by 0
  22. 2025Fundamental Limits of Crystalline Equivariant Graph Neural Networks: A Circuit Complexity Perspectivecited by 0
  23. 2025Conditional Clifford-Steerable CNNs for PDE Modelingcited by 0
  24. 2025Soft Geometric Inductive Bias for Object Centric Dynamicscited by 0
  25. 2026On Lie Groups Preserving Subspaces of Degenerate Clifford Algebrascited by 0
  26. 2026A Geometric Algebra-informed NeRF Framework for Generalizable Wireless Channel Predictioncited by 0
  27. 2026Structure-guided molecular design with contrastive 3D protein-ligand learningcited by 0
  28. 2026When Does Equivariance Help? Canonical Alignment in Neural Fluid Surrogatescited by 0
  29. 2026Velocityformer: Broken-Symmetry-Matched Equivariant Graph Transformers for Cosmological Velocity Reconstructioncited by 0
  30. 2026The Token Is a Group Element: On Lie-Algebra Attention over Matrix Lie Groupscited by 0
  31. 2026What Do Lorentz-Equivariant Jet Taggers Learn?cited by 0
  32. 2026Rotational Equivariance in Machine Learning: A Comprehensive Tutorialcited by 0
  33. 2026Screw Attention: Rigid-Body Algebra Inside a Transformercited by 0
  34. 2026Clifford Sheaf Neural Networkscited by 0
  35. 2026Coordinate-Free Automatic Differentiationcited by 0
  36. 2026RFWeave: Learning Wireless Propagation Across Scenes and Transceiver Configurationscited by 0