# Generating particle physics Lagrangians with transformers

arXiv: https://arxiv.org/abs/2501.09729 · HTML: https://mlsanity.com/paper/2501.09729/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 61 references that are in the map.
- First posted on arXiv 16 Jan 2025.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Discovering Symbolic Models from Deep Learning with Inductive Biases](https://mlsanity.com/paper/2006.11287/) (2020)
- [Particle Transformer for Jet Tagging](https://mlsanity.com/paper/2202.03772/) (2022)
- [Rediscovering orbital mechanics with machine learning](https://mlsanity.com/paper/2202.02306/) (2022)
- [Length Generalization in Arithmetic Transformers](https://mlsanity.com/paper/2306.15400/) (2023)
- [Linear algebra with transformers](https://mlsanity.com/paper/2112.01898/) (2021)
- [Lorentz-Equivariant Geometric Algebra Transformers for High-Energy Physics](https://mlsanity.com/paper/2405.14806/) (2024)
- [Learning the greatest common divisor: explaining transformer predictions](https://mlsanity.com/paper/2308.15594/) (2023)
- [A Lorentz-Equivariant Transformer for All of the LHC](https://mlsanity.com/paper/2411.00446/) (2024)
- [xVal: A Continuous Numerical Tokenization for Scientific Language Models](https://mlsanity.com/paper/2310.02989/) (2023)
- [Point Cloud Transformers applied to Collider Physics](https://mlsanity.com/paper/2102.05073/) (2021)
- [What is my math transformer doing? -- Three results on interpretability and generalization](https://mlsanity.com/paper/2211.00170/) (2022)
- [Transforming the Bootstrap: Using Transformers to Compute Scattering Amplitudes in Planar N = 4 Super Yang-Mills Theory](https://mlsanity.com/paper/2405.06107/) (2024)
- [Simplifying Polylogarithms with Machine Learning](https://mlsanity.com/paper/2206.04115/) (2022)
- [SYMBA: Symbolic Computation of Squared Amplitudes in High Energy Physics with Machine Learning](https://mlsanity.com/paper/2206.08901/) (2022)
- [Exploring the flavor structure of quarks and leptons with reinforcement learning](https://mlsanity.com/paper/2304.14176/) (2023)
- [Heterotic String Model Building with Monad Bundles and Reinforcement Learning](https://mlsanity.com/paper/2108.07316/) (2021)
- [nu2-Flows: Fast and improved neutrino reconstruction in multi-neutrino final states with conditional normalizing flows](https://mlsanity.com/paper/2307.02405/) (2023)
- [Reinforcement learning-based statistical search strategy for an axion model from flavor](https://mlsanity.com/paper/2409.10023/) (2024)
- [Contextual Counting: A Mechanistic Study of Transformers on a Quantitative Task](https://mlsanity.com/paper/2406.02585/) (2024)
- [GANs for generating EFT models](https://mlsanity.com/paper/1809.02612/) (2018)
- [PIPPIN: Generating variable length full events from partons](https://mlsanity.com/paper/2406.13074/) (2024)
- [Learning the Simplicity of Scattering Amplitudes](https://mlsanity.com/paper/2408.04720/) (2024)
- [TrackFormers: In Search of Transformer-Based Particle Tracking for the High-Luminosity LHC Era](https://mlsanity.com/paper/2407.07179/) (2024)
- [How Effective is Byte Pair Encoding for Out-Of-Vocabulary Words in Neural Machine Translation?](https://mlsanity.com/paper/2208.05225/) (2022)
- [Equivariant Transformer is all you need](https://mlsanity.com/paper/2310.13222/) (2023)
- [Induced Generative Adversarial Particle Transformers](https://mlsanity.com/paper/2312.04757/) (2023)

## Built on by (most cited first, 1 of 1)

- [Particle-Lund Multimodality in Jet Taggers](https://mlsanity.com/paper/2605.26821/) (2026)

## Abstract (arXiv)

In physics, Lagrangians provide a systematic way to describe laws governing physical systems. In the context of particle physics, they encode the interactions and behavior of the fundamental building blocks of our universe. By treating Lagrangians as complex, rule-based constructs similar to linguistic expressions, we trained a transformer model -- proven to be effective in natural language tasks -- to predict the Lagrangian corresponding to a given list of particles. We report on the transformer's performance in constructing Lagrangians respecting the Standard Model SU(3)× SU(2)× U(1) gauge symmetries. The resulting model is shown to achieve high accuracies (over 90%) with Lagrangians up to six matter fields, with the capacity to generalize beyond the training distribution, albeit within architectural constraints. We show through an analysis of input embeddings that the model has internalized concepts such as group representations and conjugation operations as it learned to generate Lagrangians. We make the model and training datasets available to the community. An interactive demonstration can be found at: urlhttps://huggingface.co/spaces/JoseEliel/generate-lagrangians.