# Aspen Open Jets: Unlocking LHC Data for Foundation Models in Particle Physics

arXiv: https://arxiv.org/abs/2412.10504 · HTML: https://mlsanity.com/paper/2412.10504/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 25 references that are in the map.
- First posted on arXiv 13 Dec 2024.

## Builds on (most cited first, 8 of 8)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Straightening Out the Straight-Through Estimator: Overcoming Optimization Challenges in Vector Quantized Networks](https://mlsanity.com/paper/2305.08842/) (2023)

## Built on by (most cited first, 14 of 14)

- [OmniJet-αC: Learning point cloud calorimeter simulations using generative transformers](https://mlsanity.com/paper/2501.05534/) (2025)
- [Neural Scaling Laws for Boosted Jet Tagging](https://mlsanity.com/paper/2602.15781/) (2026)
- [Foundation models for high-energy physics](https://mlsanity.com/paper/2509.21434/) (2025)
- [Fine-tuning machine-learned particle-flow reconstruction for new detector geometries in future colliders](https://mlsanity.com/paper/2503.00131/) (2025)
- [EveNet: A Foundation Model for Particle Collision Data Analysis](https://mlsanity.com/paper/2601.17126/) (2026)
- [Enhancing next token prediction based pre-training for jet foundation models](https://mlsanity.com/paper/2512.04149/) (2025)
- [Prompting Particle Physics: Tokenized Multi-modal Foundation Models for Combinatorially Many Tasks](https://mlsanity.com/paper/2609.31862/) (2026)
- [A Step Toward Interpretability: Smearing the Likelihood](https://mlsanity.com/paper/2501.07643/) (2025)
- [BitHEP -- The Limits of Low-Precision ML in HEP](https://mlsanity.com/paper/2504.03387/) (2025)
- [Multimodal Generative Flows for LHC Jets](https://mlsanity.com/paper/2509.01736/) (2025)
- [KIGNet: Physics-Motivated Multi-Graph Representation Learning for Explainable Jet Tagging](https://mlsanity.com/paper/2512.07420/) (2025)
- [Machine-learned particle flow as a foundation model for collider physics](https://mlsanity.com/paper/2606.14373/) (2026)
- [Scaling Collider Event Generation with Residual-Quantized Tokens](https://mlsanity.com/paper/2610.00569/) (2026)
- [Learning Standard Model structure from LHC data with Riemannian flow matching](https://mlsanity.com/paper/2607.16144/) (2026)

## Abstract (arXiv)

Foundation models are deep learning models pre-trained on large amounts of data which are capable of generalizing to multiple datasets and/or downstream tasks. This work demonstrates how data collected by the CMS experiment at the Large Hadron Collider can be useful in pre-training foundation models for HEP. Specifically, we introduce the AspenOpenJets dataset, consisting of approximately 178M high pT jets derived from CMS 2016 Open Data. We show how pre-training the OmniJet-α foundation model on AspenOpenJets improves performance on generative tasks with significant domain shift: generating boosted top and QCD jets from the simulated JetClass dataset. In addition to demonstrating the power of pre-training of a jet-based foundation model on actual proton-proton collision data, we provide the ML-ready derived AspenOpenJets dataset for further public use.