# A Lorentz-Equivariant Transformer for All of the LHC

arXiv: https://arxiv.org/abs/2411.00446 · HTML: https://mlsanity.com/paper/2411.00446/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 85 references that are in the map.
- First posted on arXiv 1 Nov 2024.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Stochastic Interpolants: A Unifying Framework for Flows and Diffusions](https://mlsanity.com/paper/2303.08797/) (2023)
- [On Layer Normalization in the Transformer Architecture](https://mlsanity.com/paper/2002.04745/) (2020)
- [Symbolic Discovery of Optimization Algorithms](https://mlsanity.com/paper/2302.06675/) (2023)
- [Scalars are universal: Equivariant machine learning, structured like classical physics](https://mlsanity.com/paper/2106.06610/) (2021)
- [Particle Transformer for Jet Tagging](https://mlsanity.com/paper/2202.03772/) (2022)
- [Energy Flow Networks: Deep Sets for Particle Jets](https://mlsanity.com/paper/1810.05165/) (2018)
- [CaloGAN: Simulating 3D High Energy Particle Showers in Multi-Layer Electromagnetic Calorimeters with Generative Adversarial Networks](https://mlsanity.com/paper/1712.10321/) (2017)
- [ParticleNet: Jet Tagging via Particle Clouds](https://mlsanity.com/paper/1902.08570/) (2019)
- [Clifford Group Equivariant Neural Networks](https://mlsanity.com/paper/2305.11141/) (2023)
- [Geometric Algebra Transformer](https://mlsanity.com/paper/2305.18415/) (2023)
- [Jet-Images -- Deep Learning Edition](https://mlsanity.com/paper/1511.05190/) (2015)
- [Lorentz-Equivariant Geometric Algebra Transformers for High-Energy Physics](https://mlsanity.com/paper/2405.14806/) (2024)
- [CaloChallenge 2022: A Community Challenge for Fast Calorimeter Simulation](https://mlsanity.com/paper/2410.21611/) (2024)
- [How to GAN away Detector Effects](https://mlsanity.com/paper/1912.00477/) (2019)
- [LHC analysis-specific datasets with Generative Adversarial Networks](https://mlsanity.com/paper/1901.05282/) (2019)
- [PELICAN: Permutation Equivariant and Lorentz Invariant or Covariant Aggregator Network for Particle Physics](https://mlsanity.com/paper/2211.00454/) (2022)
- [Explainable Equivariant Neural Networks for Particle Physics: PELICAN](https://mlsanity.com/paper/2307.16506/) (2023)
- [QCD-Aware Recursive Neural Networks for Jet Physics](https://mlsanity.com/paper/1702.00748/) (2017)
- [End-To-End Latent Variational Diffusion Models for Inverse Problems in High Energy Physics](https://mlsanity.com/paper/2305.10399/) (2023)
- [An unfolding method based on conditional Invertible Neural Networks (cINN) using iterative training](https://mlsanity.com/paper/2212.08674/) (2022)
- [CaloClouds II: Ultra-Fast Geometry-Independent Highly-Granular Calorimeter Simulation](https://mlsanity.com/paper/2309.05704/) (2023)

## Built on by (most cited first, 28 of 28)

- [Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Models](https://mlsanity.com/paper/2501.05382/) (2025)
- [OmniJet-αC: Learning point cloud calorimeter simulations using generative transformers](https://mlsanity.com/paper/2501.05534/) (2025)
- [Lorentz Local Canonicalization: How to Make Any Network Lorentz-Equivariant](https://mlsanity.com/paper/2505.20280/) (2025)
- [Neural Scaling Laws for Boosted Jet Tagging](https://mlsanity.com/paper/2602.15781/) (2026)
- [Foundation models for high-energy physics](https://mlsanity.com/paper/2509.21434/) (2025)
- [CaloHadronic: a diffusion model for the generation of hadronic showers](https://mlsanity.com/paper/2506.21720/) (2025)
- [Generative Unfolding with Distribution Mapping](https://mlsanity.com/paper/2411.02495/) (2024)
- [RINO: Renormalization Group Invariance with No Labels](https://mlsanity.com/paper/2509.07486/) (2025)
- [Scaling laws for amplitude surrogates](https://mlsanity.com/paper/2601.13308/) (2026)
- [Hybrid Diffusion Policies with Projective Geometric Algebra for Efficient Robot Manipulation Learning](https://mlsanity.com/paper/2507.05695/) (2025)
- [Forecasting Generative Amplification](https://mlsanity.com/paper/2509.08048/) (2025)
- [Enhancing next token prediction based pre-training for jet foundation models](https://mlsanity.com/paper/2512.04149/) (2025)
- [AllShowers: One model for all calorimeter showers](https://mlsanity.com/paper/2601.11716/) (2026)
- [Extrapolating Jet Radiation with Autoregressive Transformers](https://mlsanity.com/paper/2412.12074/) (2024)
- [Stable and Interpretable Jet Physics with IRC-Safe Equivariant Feature Extraction](https://mlsanity.com/paper/2509.22059/) (2025)
- [A First Full Physics Benchmark for Highly Granular Calorimeter Surrogates](https://mlsanity.com/paper/2511.17293/) (2025)
- [Approximate Equivariance via Projection-based Regularisation](https://mlsanity.com/paper/2601.05028/) (2026)
- [jBOT: Semantic Jet Representation Clustering Emerges from Self-Distillation](https://mlsanity.com/paper/2601.11719/) (2026)
- [Generating particle physics Lagrangians with transformers](https://mlsanity.com/paper/2501.09729/) (2025)
- [BitHEP -- The Limits of Low-Precision ML in HEP](https://mlsanity.com/paper/2504.03387/) (2025)
- [Latent Representation Learning in Heavy-Ion Collisions with MaskPoint Transformer](https://mlsanity.com/paper/2510.06691/) (2025)
- [Pointwise Metrics Mislead: An Evaluation Protocol for Multimodal Inverse Problems](https://mlsanity.com/paper/2605.22891/) (2026)
- [Unknown Unknowns: Model Misspecification in Machine Learning for Physics](https://mlsanity.com/paper/2608.13633/) (2026)
- [Learnable cut flow for high energy physics](https://mlsanity.com/paper/2503.22498/) (2025)
- [Softly Induced Functional Simplicity: Implications for Neural Network Generalisation, Robustness, and Distillation](https://mlsanity.com/paper/2601.06584/) (2026)
- [Event Generation with Parallel Langevin Sampling and Learned Stein Diagnostics](https://mlsanity.com/paper/2606.14854/) (2026)
- [Machine learning for the LHC physics program: a 2025-2026 stocktake](https://mlsanity.com/paper/2609.32874/) (2026)
- [RINO: Renormalization Group Distillation with No Labels](https://mlsanity.com/paper/2610.15919/) (2026)

## Abstract (arXiv)

We show that the Lorentz-Equivariant Geometric Algebra Transformer (L-GATr) yields state-of-the-art performance for a wide range of machine learning tasks at the Large Hadron Collider. L-GATr represents data in a geometric algebra over space-time and is equivariant under Lorentz transformations. The underlying architecture is a versatile and scalable transformer, which is able to break symmetries if needed. We demonstrate the power of L-GATr for amplitude regression and jet classification, and then benchmark it as the first Lorentz-equivariant generative network. For all three LHC tasks, we find significant improvements over previous architectures.