mlsanityData to 7 Oct 2026

Aspen Open Jets: Unlocking LHC Data for Foundation Models in Particle Physics

arXiv 2412.10504 ↗First posted 13 Dec 2024hep-ph cs.LG hep-excited by 14

Not on the map. Why?

Abstract (arXiv)

Foundation models are deep learning models pre-trained on large amounts of data which are capable of generalizing to multiple datasets and/or downstream tasks. This work demonstrates how data collected by the CMS experiment at the Large Hadron Collider can be useful in pre-training foundation models for HEP. Specifically, we introduce the AspenOpenJets dataset, consisting of approximately 178M high pT jets derived from CMS 2016 Open Data. We show how pre-training the OmniJet-α foundation model on AspenOpenJets improves performance on generative tasks with significant domain shift: generating boosted top and QCD jets from the simulated JetClass dataset. In addition to demonstrating the power of pre-training of a jet-based foundation model on actual proton-proton collision data, we provide the ML-ready derived AspenOpenJets dataset for further public use.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 8 of its 25 references that are in the map. First posted on arXiv 13 Dec 2024.

Builds on

8 of 25 references.

References · 8

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  6. 2017Neural Discrete Representation Learningcited by 4,650
  7. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  8. 2023Straightening Out the Straight-Through Estimator: Overcoming Optimization Challenges in Vector Quantized Networkscited by 80

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 6.

14 papers.

Sort:

  1. 2025OmniJet-αC: Learning point cloud calorimeter simulations using generative transformerscited by 14
  2. 2026Neural Scaling Laws for Boosted Jet Taggingcited by 11
  3. 2025Foundation models for high-energy physicscited by 10
  4. 2025Fine-tuning machine-learned particle-flow reconstruction for new detector geometries in future colliderscited by 8
  5. 2026EveNet: A Foundation Model for Particle Collision Data Analysiscited by 8
  6. 2025Enhancing next token prediction based pre-training for jet foundation modelscited by 3
  7. 2025A Step Toward Interpretability: Smearing the Likelihoodcited by 1
  8. 2025BitHEP -- The Limits of Low-Precision ML in HEPcited by 1
  9. 2025Multimodal Generative Flows for LHC Jetscited by 1
  10. 2025KIGNet: Physics-Motivated Multi-Graph Representation Learning for Explainable Jet Taggingcited by 1
  11. 2026Machine-learned particle flow as a foundation model for collider physicscited by 1
  12. 2026Prompting Particle Physics: Tokenized Multi-modal Foundation Models for Combinatorially Many Taskscited by 1
2 more
  1. 2026Learning Standard Model structure from LHC data with Riemannian flow matchingcited by 0
  2. 2026Scaling Collider Event Generation with Residual-Quantized Tokenscited by 0