# HEP-JEPA: A foundation model for collider physics using joint embedding predictive architecture

arXiv: https://arxiv.org/abs/2502.03933 · HTML: https://mlsanity.com/paper/2502.03933/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Feb 2025.

## Built on by (most cited first, 13 of 13)

- [RINO: Renormalization Group Invariance with No Labels](https://mlsanity.com/paper/2509.07486/) (2025)
- [AeroJEPA: Learning Semantic Latent Representations for Scalable 3D Aerodynamic Field Modeling](https://mlsanity.com/paper/2605.05586/) (2026)
- [Enhancing next token prediction based pre-training for jet foundation models](https://mlsanity.com/paper/2512.04149/) (2025)
- [Cross-Domain Transfer with Particle Physics Foundation Models: From Jets to Neutrino Interactions](https://mlsanity.com/paper/2604.12364/) (2026)
- [JetParticle-JEPA: An Efficient Self-Supervised Representation Learning method for Jet Tagging in High-Energy Physics](https://mlsanity.com/paper/2606.14813/) (2026)
- [Towards Engineering Scaling Laws with Pretraining Data Composition](https://mlsanity.com/paper/2606.19781/) (2026)
- [Prompting Particle Physics: Tokenized Multi-modal Foundation Models for Combinatorially Many Tasks](https://mlsanity.com/paper/2609.31862/) (2026)
- [Application of a Mixture of Experts-based Foundation Model to the GlueX DIRC Detector](https://mlsanity.com/paper/2604.24775/) (2026)
- [Pre-Training for Simulation-Based Science: A Study on Jet Foundation Model Training Objectives](https://mlsanity.com/paper/2606.14870/) (2026)
- [Tagging fully hadronic exotic decays of the vectorlike B quark using a graph neural network](https://mlsanity.com/paper/2505.07769/) (2025)
- [Patch Hierarchical Attention Transformer for Efficient Particle Jet Tagging](https://mlsanity.com/paper/2605.21789/) (2026)
- [Local Diagnostics of Learned Event Representations for Anomaly Detection at the HL-LHC](https://mlsanity.com/paper/2610.13437/) (2026)
- [RINO: Renormalization Group Distillation with No Labels](https://mlsanity.com/paper/2610.15919/) (2026)

## Abstract (arXiv)

We present a transformer architecture-based foundation model for tasks at high-energy particle colliders such as the Large Hadron Collider. We train the model to classify jets using a self-supervised strategy inspired by the Joint Embedding Predictive Architecture. We use the JetClass dataset containing 100M jets of various known particles to pre-train the model with a data-centric approach -- the model uses a fraction of the jet constituents as the context to predict the embeddings of the unseen target constituents. Our pre-trained model fares well with other datasets for standard classification benchmark tasks. We test our model on two additional downstream tasks: top tagging and differentiating light-quark jets from gluon jets. We also evaluate our model with task-specific metrics and baselines and compare it with state-of-the-art models in high-energy physics. Project site: https://hep-jepa.github.io/