# FM4NPP: A Scaling Foundation Model for Nuclear and Particle Physics

arXiv: https://arxiv.org/abs/2508.14087 · HTML: https://mlsanity.com/paper/2508.14087/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 40 references that are in the map.
- First posted on arXiv 13 Aug 2025.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Interaction Networks for Learning about Objects, Relations and Physics](https://mlsanity.com/paper/1612.00222/) (2016)
- [InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions](https://mlsanity.com/paper/2211.05778/) (2022)
- [Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer](https://mlsanity.com/paper/2203.03466/) (2022)
- [Sensitivity and Generalization in Neural Networks: an Empirical Study](https://mlsanity.com/paper/1802.08760/) (2018)
- [Learning representations of irregular particle-detector geometry with distance-weighted graph networks](https://mlsanity.com/paper/1902.07987/) (2019)
- [Performance of a Geometric Deep Learning Pipeline for HL-LHC Particle Tracking](https://mlsanity.com/paper/2103.06995/) (2021)
- [Progress and Opportunities of Foundation Models in Bioinformatics](https://mlsanity.com/paper/2402.04286/) (2024)
- [Machine Learning Algorithms for b-Jet Tagging at the ATLAS Experiment](https://mlsanity.com/paper/1711.08811/) (2017)
- [EggNet: An Evolving Graph-based Graph Attention Network for Particle Track Reconstruction](https://mlsanity.com/paper/2407.13925/) (2024)

## Built on by (most cited first, 6 of 6)

- [EveNet: A Foundation Model for Particle Collision Data Analysis](https://mlsanity.com/paper/2601.17126/) (2026)
- [Cross-Domain Transfer with Particle Physics Foundation Models: From Jets to Neutrino Interactions](https://mlsanity.com/paper/2604.12364/) (2026)
- [Panda: Self-distillation of Reusable Sensor-level Representations for High Energy Physics](https://mlsanity.com/paper/2512.01324/) (2025)
- [Towards Engineering Scaling Laws with Pretraining Data Composition](https://mlsanity.com/paper/2606.19781/) (2026)
- [Panda Diplomacy: Foundation Model Pre-training across Particle Imaging Detectors for High Energy and Nuclear Physics](https://mlsanity.com/paper/2609.00611/) (2026)
- [How to scale your HEP ML models: A recipe for robust architecture comparisons at scale](https://mlsanity.com/paper/2610.06784/) (2026)

## Abstract (arXiv)

Large language models have revolutionized artificial intelligence by enabling large, generalizable models trained through self-supervision. This paradigm has inspired the development of scientific foundation models (FMs). However, applying this capability to experimental particle physics is challenging due to the sparse, spatially distributed nature of detector data, which differs dramatically from natural language. This work addresses if an FM for particle physics can scale and generalize across diverse tasks. We introduce a new dataset with more than 11 million particle collision events and a suite of downstream tasks and labeled data for evaluation. We propose a novel self-supervised training method for detector data and demonstrate its neural scalability with models that feature up to 188 million parameters. With frozen weights and task-specific adapters, this FM consistently outperforms baseline models across all downstream tasks. The performance also exhibits robust data-efficient adaptation. Further analysis reveals that the representations extracted by the FM are task-agnostic but can be specialized via a single linear mapping for different downstream tasks.