mlsanityData to 7 Oct 2026

Euclidean, Projective, Conformal: Choosing a Geometric Algebra for Equivariant Transformers

arXiv 2311.04744 ↗First posted 8 Nov 2023cs.LG cs.AIcited by 12

Not on the map. Why?

Abstract (arXiv)

The Geometric Algebra Transformer (GATr) is a versatile architecture for geometric deep learning based on projective geometric algebra. We generalize this architecture into a blueprint that allows one to construct a scalable transformer architecture given any geometric (or Clifford) algebra. We study versions of this architecture for Euclidean, projective, and conformal algebras, all of which are suited to represent 3D data, and evaluate them in theory and practice. The simplest Euclidean architecture is computationally cheap, but has a smaller symmetry group and is not as sample-efficient, while the projective model is not sufficiently expressive. Both the conformal algebra and an improved version of the projective algebra define powerful, performant architectures.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 19 of its 41 references that are in the map. First posted on arXiv 8 Nov 2023.

Builds on

19 of 41 references.

References · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  4. 2016Group Equivariant Convolutional Networkscited by 1,154
  5. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  6. 2021E(n) Equivariant Graph Neural Networkscited by 951
  7. 2020On Layer Normalization in the Transformer Architecturecited by 906
  8. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 735
  9. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 602
  10. 2022MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fieldscited by 467
  11. 2022Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphscited by 258
  12. 2021Geometric and Physical Quantities Improve E(3) Equivariant Message Passingcited by 225
7 more
  1. 2021A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groupscited by 185
  2. 2021Self-attention Does Not Need O(n2) Memorycited by 152
  3. 2022Clifford Neural Layers for PDE Modelingcited by 91
  4. 2020Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphscited by 90
  5. 2023Clifford Group Equivariant Neural Networkscited by 54
  6. 2022So3krates: Equivariant attention for interactions on arbitrary length-scales in molecular systemscited by 39
  7. 2022Mesh Neural Networks for SE(3)-Equivariant Hemodynamics Estimation on the Artery Wallcited by 29

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 2; 2026: 2.

12 papers.

  1. 2024Does equivariance matter at scale?cited by 49
  2. 2024Lorentz-Equivariant Geometric Algebra Transformers for High-Energy Physicscited by 36
  3. 2024A Lorentz-Equivariant Transformer for All of the LHCcited by 27
  4. 2024Morphological Symmetries in Roboticscited by 9
  5. 2024SE(3)-Hyena Operator for Scalable Equivariant Learningcited by 6
  6. 2025Geometric Hyena Networks for Large-scale Equivariant Learningcited by 4
  7. 2025Hybrid Diffusion Policies with Projective Geometric Algebra for Efficient Robot Manipulation Learningcited by 3
  8. 2024Randomized Geometric Algebra Methods for Convex Neural Networkscited by 1
  9. 2024Optimal Symmetries in Binary Classificationcited by 1
  10. 2024Harmformer: Harmonic Networks Meet Transformers for Continuous Roto-Translation Equivariancecited by 1
  11. 2026The Token Is a Group Element: On Lie-Algebra Attention over Matrix Lie Groupscited by 0
  12. 2026SKooP: Symmetric Koopman Predictions for Faster and More Generalizable Legged Robot Locomotion with Reinforcement Learningcited by 0