# Foundation models for high-energy physics

arXiv: https://arxiv.org/abs/2509.21434 · HTML: https://mlsanity.com/paper/2509.21434/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 52 references that are in the map.
- First posted on arXiv 25 Sep 2025.

## Builds on (most cited first, 19 of 19)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Multimodal Foundation Models: From Specialists to General-Purpose Assistants](https://mlsanity.com/paper/2309.10020/) (2023)
- [Particle Transformer for Jet Tagging](https://mlsanity.com/paper/2202.03772/) (2022)
- [Straightening Out the Straight-Through Estimator: Overcoming Optimization Challenges in Vector Quantized Networks](https://mlsanity.com/paper/2305.08842/) (2023)
- [Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Models](https://mlsanity.com/paper/2401.13537/) (2024)
- [OmniJet-α: The first cross-task foundation model for particle physics](https://mlsanity.com/paper/2403.05618/) (2024)
- [A Lorentz-Equivariant Transformer for All of the LHC](https://mlsanity.com/paper/2411.00446/) (2024)
- [Jet tagging in the Lund plane with graph networks](https://mlsanity.com/paper/2012.08526/) (2020)
- [Re-Simulation-based Self-Supervised Learning for Pre-Training Foundation Models](https://mlsanity.com/paper/2403.07066/) (2024)
- [Is Tokenization Needed for Masked Particle Modelling?](https://mlsanity.com/paper/2409.12589/) (2024)
- [CaloClouds II: Ultra-Fast Geometry-Independent Highly-Granular Calorimeter Simulation](https://mlsanity.com/paper/2309.05704/) (2023)
- [Finetuning Foundation Models for Joint Analysis Optimization](https://mlsanity.com/paper/2401.13536/) (2024)
- [Aspen Open Jets: Unlocking LHC Data for Foundation Models in Particle Physics](https://mlsanity.com/paper/2412.10504/) (2024)
- [OmniJet-αC: Learning point cloud calorimeter simulations using generative transformers](https://mlsanity.com/paper/2501.05534/) (2025)
- [Convolutional L2LFlows: Generating Accurate Showers in Highly Granular Calorimeters Using Convolutional Normalizing Flows](https://mlsanity.com/paper/2405.20407/) (2024)
- [Learning Symmetry-Independent Jet Representations via Jet-Based Joint Embedding Predictive Architecture](https://mlsanity.com/paper/2412.05333/) (2024)
- [Bumblebee: Foundation Model for Particle Physics Discovery](https://mlsanity.com/paper/2412.07867/) (2024)
- [Pretrained Event Classification Model for High Energy Physics Analysis](https://mlsanity.com/paper/2412.10665/) (2024)

## Built on by (most cited first, 10 of 10)

- [Cross-Domain Transfer with Particle Physics Foundation Models: From Jets to Neutrino Interactions](https://mlsanity.com/paper/2604.12364/) (2026)
- [jBOT: Semantic Jet Representation Clustering Emerges from Self-Distillation](https://mlsanity.com/paper/2601.11719/) (2026)
- [Towards Engineering Scaling Laws with Pretraining Data Composition](https://mlsanity.com/paper/2606.19781/) (2026)
- [Are We Ready for AI-Driven Discovery? AI Verification Before the Next Fundamental Physics Breakthrough](https://mlsanity.com/paper/2607.10039/) (2026)
- [Prompting Particle Physics: Tokenized Multi-modal Foundation Models for Combinatorially Many Tasks](https://mlsanity.com/paper/2609.31862/) (2026)
- [Machine-learned particle flow as a foundation model for collider physics](https://mlsanity.com/paper/2606.14373/) (2026)
- [Learning Standard Model structure from LHC data with Riemannian flow matching](https://mlsanity.com/paper/2607.16144/) (2026)
- [Similarity Pairing with Energy Mover's Distance for Self-Supervised Pre-Training at the LHC](https://mlsanity.com/paper/2609.17738/) (2026)
- [Machine learning for the LHC physics program: a 2025-2026 stocktake](https://mlsanity.com/paper/2609.32874/) (2026)
- [Machine Learning Meets High-Energy Nuclear Physics: From Pattern Recognition to Physics-Integrated Discovery](https://mlsanity.com/paper/2610.12293/) (2026)

## Abstract (arXiv)

The rise of foundation models -- large, pretrained machine learning models that can be finetuned to a variety of tasks -- has revolutionized the fields of natural language processing and computer vision. In high-energy physics, the question of whether these models can be implemented directly in physics research, or even built from scratch, tailored for particle physics data, has generated an increasing amount of attention. This review, which is the first on the topic of foundation models in high-energy physics, summarizes and discusses the research that has been published in the field so far.