# Cross-Domain Transfer with Particle Physics Foundation Models: From Jets to Neutrino Interactions

arXiv: https://arxiv.org/abs/2604.12364 · HTML: https://mlsanity.com/paper/2604.12364/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 66 references that are in the map.
- First posted on arXiv 14 Apr 2026.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Transfusion: Understanding Transfer Learning for Medical Imaging](https://mlsanity.com/paper/1902.07208/) (2019)
- [A Living Review of Machine Learning for Particle Physics](https://mlsanity.com/paper/2102.02770/) (2021)
- [Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics](https://mlsanity.com/paper/2110.01518/) (2021)
- [Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Models](https://mlsanity.com/paper/2401.13537/) (2024)
- [OmniJet-α: The first cross-task foundation model for particle physics](https://mlsanity.com/paper/2403.05618/) (2024)
- [Is Tokenization Needed for Masked Particle Modelling?](https://mlsanity.com/paper/2409.12589/) (2024)
- [HEP-JEPA: A foundation model for collider physics using joint embedding predictive architecture](https://mlsanity.com/paper/2502.03933/) (2025)
- [Foundation models for high-energy physics](https://mlsanity.com/paper/2509.21434/) (2025)
- [Automating High Energy Physics Data Analysis with LLM-Powered Agents](https://mlsanity.com/paper/2512.07785/) (2025)
- [Particle Trajectory Representation Learning with Masked Point Modeling](https://mlsanity.com/paper/2502.02558/) (2025)
- [Xiwu: A Basis Flexible and Learnable LLM for High Energy Physics](https://mlsanity.com/paper/2404.08001/) (2024)
- [FM4NPP: A Scaling Foundation Model for Nuclear and Particle Physics](https://mlsanity.com/paper/2508.14087/) (2025)
- [Panda: Self-distillation of Reusable Sensor-level Representations for High Energy Physics](https://mlsanity.com/paper/2512.01324/) (2025)

## Built on by (most cited first, 3 of 3)

- [Pre-Training for Simulation-Based Science: A Study on Jet Foundation Model Training Objectives](https://mlsanity.com/paper/2606.14870/) (2026)
- [Panda Diplomacy: Foundation Model Pre-training across Particle Imaging Detectors for High Energy and Nuclear Physics](https://mlsanity.com/paper/2609.00611/) (2026)
- [Machine learning for the LHC physics program: a 2025-2026 stocktake](https://mlsanity.com/paper/2609.32874/) (2026)

## Abstract (arXiv)

Future AI-based studies in particle physics will likely start from a foundation model to accelerate training and enhance sensitivity. As a step toward a general-purpose foundation model for particle physics, we investigate whether the OmniLearned and ParticleViT foundation models pretrained on diverse high-Q2 simulated and real pp and ep collisions retain useful knowledge to a few-GeV fixed-target neutrino experiment. We process MINERvA neutrino--nucleus scattering events and evaluate pretrained models on two types of tasks: regression of available energy and binary classification of charged-current pion final states (mathrmCC1π±, mathrmCCNπ±, and mathrmCC1π0). Pretrained OmniLearned and ParticleViT models outperform similarly sized models trained from scratch at the same compute budget, with the largest gains for OmniLearned on regression and for ParticleViT on classification. When the same transformer architecture is instead initialized from unrelated text pretraining (BERT), this advantage appears only marginally for classification in terms of compute efficiency and not in any way for regression. These results suggest that particle-level foundation models acquire inductive biases that generalize across large differences in energy scale, detector technology, and underlying physics processes, pointing toward detector-agnostic inference in particle physics.