# Lorentz-Equivariant Geometric Algebra Transformers for High-Energy Physics

arXiv: https://arxiv.org/abs/2405.14806 · HTML: https://mlsanity.com/paper/2405.14806/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 75 references that are in the map.
- First posted on arXiv 23 May 2024.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Stochastic Interpolants: A Unifying Framework for Flows and Diffusions](https://mlsanity.com/paper/2303.08797/) (2023)
- [On Layer Normalization in the Transformer Architecture](https://mlsanity.com/paper/2002.04745/) (2020)
- [Equivariant Diffusion for Molecule Generation in 3D](https://mlsanity.com/paper/2203.17003/) (2022)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [Revisiting Classifier Two-Sample Tests](https://mlsanity.com/paper/1610.06545/) (2016)
- [A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups](https://mlsanity.com/paper/2104.09459/) (2021)
- [Scalars are universal: Equivariant machine learning, structured like classical physics](https://mlsanity.com/paper/2106.06610/) (2021)
- [Particle Transformer for Jet Tagging](https://mlsanity.com/paper/2202.03772/) (2022)
- [Clifford Neural Layers for PDE Modeling](https://mlsanity.com/paper/2209.04934/) (2022)
- [Energy Flow Networks: Deep Sets for Particle Jets](https://mlsanity.com/paper/1810.05165/) (2018)
- [Clifford Group Equivariant Neural Networks](https://mlsanity.com/paper/2305.11141/) (2023)
- [Jet-Images -- Deep Learning Edition](https://mlsanity.com/paper/1511.05190/) (2015)
- [Geometric Clifford Algebra Networks](https://mlsanity.com/paper/2302.06594/) (2023)
- [Constraining Effective Field Theories with Machine Learning](https://mlsanity.com/paper/1805.00013/) (2018)
- [PELICAN: Permutation Equivariant and Lorentz Invariant or Covariant Aggregator Network for Particle Physics](https://mlsanity.com/paper/2211.00454/) (2022)
- [Clifford-Steerable Convolutional Neural Networks](https://mlsanity.com/paper/2402.14730/) (2024)
- [Clifford Group Equivariant Simplicial Message Passing Networks](https://mlsanity.com/paper/2402.10011/) (2024)
- [Generative Networks for Precision Enthusiasts](https://mlsanity.com/paper/2110.13632/) (2021)
- [The Landscape of Unfolding with Machine Learning](https://mlsanity.com/paper/2404.18807/) (2024)
- [Euclidean, Projective, Conformal: Choosing a Geometric Algebra for Equivariant Transformers](https://mlsanity.com/paper/2311.04744/) (2023)
- [Covariance in Physics and Convolutional Neural Networks](https://mlsanity.com/paper/1906.02481/) (2019)
- [Simulation-based inference methods for particle physics](https://mlsanity.com/paper/2010.06439/) (2020)
- [Graded Symmetry Groups: Plane and Simple](https://mlsanity.com/paper/2107.03771/) (2021)

## Built on by (most cited first, 30 of 38)

- [Does equivariance matter at scale?](https://mlsanity.com/paper/2410.23179/) (2024)
- [A Lorentz-Equivariant Transformer for All of the LHC](https://mlsanity.com/paper/2411.00446/) (2024)
- [Convolutional L2LFlows: Generating Accurate Showers in Highly Granular Calorimeters Using Convolutional Normalizing Flows](https://mlsanity.com/paper/2405.20407/) (2024)
- [Lorentz Local Canonicalization: How to Make Any Network Lorentz-Equivariant](https://mlsanity.com/paper/2505.20280/) (2025)
- [Generative Unfolding with Distribution Mapping](https://mlsanity.com/paper/2411.02495/) (2024)
- [RINO: Renormalization Group Invariance with No Labels](https://mlsanity.com/paper/2509.07486/) (2025)
- [Sub-microsecond Transformers for Jet Tagging on FPGAs](https://mlsanity.com/paper/2510.24784/) (2025)
- [Scaling laws for amplitude surrogates](https://mlsanity.com/paper/2601.13308/) (2026)
- [Multivector Neurons: Better and Faster O(n)-Equivariant Clifford Graph Neural Networks](https://mlsanity.com/paper/2406.04052/) (2024)
- [Optimal Equivariant Architectures from the Symmetries of Matrix-Element Likelihoods](https://mlsanity.com/paper/2410.18553/) (2024)
- [Lorentz-Equivariant Quantum Graph Neural Network for High-Energy Physics](https://mlsanity.com/paper/2411.01641/) (2024)
- [Clifford Group Equivariant Diffusion Models for 3D Molecular Generation](https://mlsanity.com/paper/2504.15773/) (2025)
- [Forecasting Generative Amplification](https://mlsanity.com/paper/2509.08048/) (2025)
- [Enhancing next token prediction based pre-training for jet foundation models](https://mlsanity.com/paper/2512.04149/) (2025)
- [JetParticle-JEPA: An Efficient Self-Supervised Representation Learning method for Jet Tagging in High-Energy Physics](https://mlsanity.com/paper/2606.14813/) (2026)
- [Extrapolating Jet Radiation with Autoregressive Transformers](https://mlsanity.com/paper/2412.12074/) (2024)
- [Simulation-based inference for Precision Neutrino Physics through Neural Monte Carlo tuning](https://mlsanity.com/paper/2507.23297/) (2025)
- [Approximate Equivariance via Projection-based Regularisation](https://mlsanity.com/paper/2601.05028/) (2026)
- [Statistical Properties of Training & Generalization](https://mlsanity.com/paper/2606.20299/) (2026)
- [Interpreting "Interpretability" and Explaining "Explainability" in Machine Learning in Physics](https://mlsanity.com/paper/2606.26228/) (2026)
- [Prompting Particle Physics: Tokenized Multi-modal Foundation Models for Combinatorially Many Tasks](https://mlsanity.com/paper/2609.31862/) (2026)
- [Generating particle physics Lagrangians with transformers](https://mlsanity.com/paper/2501.09729/) (2025)
- [Latent Representation Learning in Heavy-Ion Collisions with MaskPoint Transformer](https://mlsanity.com/paper/2510.06691/) (2025)
- [SEAL - A Symmetry EncourAging Loss for High Energy Physics](https://mlsanity.com/paper/2511.01982/) (2025)
- [Pointwise Metrics Mislead: An Evaluation Protocol for Multimodal Inverse Problems](https://mlsanity.com/paper/2605.22891/) (2026)
- [Conditional Clifford-Steerable CNNs for PDE Modeling](https://mlsanity.com/paper/2510.14007/) (2025)
- [Soft Geometric Inductive Bias for Object Centric Dynamics](https://mlsanity.com/paper/2512.15493/) (2025)
- [Comparison of Image Processing Models in Quark Gluon Jet Classification](https://mlsanity.com/paper/2602.00141/) (2026)
- [Generative models on phase space](https://mlsanity.com/paper/2604.02415/) (2026)
- [A Geometric Algebra-informed NeRF Framework for Generalizable Wireless Channel Prediction](https://mlsanity.com/paper/2604.11983/) (2026)

## Abstract (arXiv)

Extracting scientific understanding from particle-physics experiments requires solving diverse learning problems with high precision and good data efficiency. We propose the Lorentz Geometric Algebra Transformer (L-GATr), a new multi-purpose architecture for high-energy physics. L-GATr represents high-energy data in a geometric algebra over four-dimensional space-time and is equivariant under Lorentz transformations, the symmetry group of relativistic kinematics. At the same time, the architecture is a Transformer, which makes it versatile and scalable to large systems. L-GATr is first demonstrated on regression and classification tasks from particle physics. We then construct the first Lorentz-equivariant generative model: a continuous normalizing flow based on an L-GATr network, trained with Riemannian flow matching. Across our experiments, L-GATr is on par with or outperforms strong domain-specific baselines.