# Finetuning Foundation Models for Joint Analysis Optimization

arXiv: https://arxiv.org/abs/2401.13536 · HTML: https://mlsanity.com/paper/2401.13536/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 62 references that are in the map.
- First posted on arXiv 24 Jan 2024.

## Builds on (most cited first, 30 of 31)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [Lookahead Optimizer: k steps forward, 1 step back](https://mlsanity.com/paper/1907.08610/) (2019)
- [HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution](https://mlsanity.com/paper/2306.15794/) (2023)
- [Particle Transformer for Jet Tagging](https://mlsanity.com/paper/2202.03772/) (2022)
- [Energy Flow Networks: Deep Sets for Particle Jets](https://mlsanity.com/paper/1810.05165/) (2018)
- [MLPF: Efficient machine-learned particle-flow reconstruction using graph neural networks](https://mlsanity.com/paper/2101.08578/) (2021)
- [Performance of a Geometric Deep Learning Pipeline for HL-LHC Particle Tracking](https://mlsanity.com/paper/2103.06995/) (2021)
- [Disentangling Neuron Representations with Concept Vectors](https://mlsanity.com/paper/2304.09707/) (2023)
- [QCD-Aware Recursive Neural Networks for Jet Physics](https://mlsanity.com/paper/1702.00748/) (2017)
- [Charged particle tracking via edge-classifying interaction networks](https://mlsanity.com/paper/2103.16701/) (2021)
- [INFERNO: Inference-Aware Neural Optimisation](https://mlsanity.com/paper/1806.04743/) (2018)
- [A FAIR and AI-ready Higgs boson decay dataset](https://mlsanity.com/paper/2108.02214/) (2021)
- [Machine Learning for Particle Flow Reconstruction at CMS](https://mlsanity.com/paper/2203.00330/) (2022)
- [neos: End-to-End-Optimised Summary Statistics for High Energy Physics](https://mlsanity.com/paper/2203.05570/) (2022)
- [Differentiable Matrix Elements with MadJax](https://mlsanity.com/paper/2203.00057/) (2022)
- [Hierarchical Graph Neural Networks for Particle Track Reconstruction](https://mlsanity.com/paper/2303.01640/) (2023)
- [Towards a Foundation Model for Neural Network Wavefunctions](https://mlsanity.com/paper/2303.09949/) (2023)
- [Branches of a Tree: Taking Derivatives of Programs with Discrete and Branching Randomness in High Energy Physics](https://mlsanity.com/paper/2308.16680/) (2023)
- [Leveraging universality of jet taggers through transfer learning](https://mlsanity.com/paper/2203.06210/) (2022)

## Built on by (most cited first, 18 of 18)

- [OmniJet-α: The first cross-task foundation model for particle physics](https://mlsanity.com/paper/2403.05618/) (2024)
- [Re-Simulation-based Self-Supervised Learning for Pre-Training Foundation Models](https://mlsanity.com/paper/2403.07066/) (2024)
- [Is Tokenization Needed for Masked Particle Modelling?](https://mlsanity.com/paper/2409.12589/) (2024)
- [PAPERCLIP: Associating Astronomical Observations and Natural Language with Multi-Modal Models](https://mlsanity.com/paper/2403.08851/) (2024)
- [Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Models](https://mlsanity.com/paper/2501.05382/) (2025)
- [Neural Scaling Laws for Boosted Jet Tagging](https://mlsanity.com/paper/2602.15781/) (2026)
- [Foundation models for high-energy physics](https://mlsanity.com/paper/2509.21434/) (2025)
- [Fine-tuning machine-learned particle-flow reconstruction for new detector geometries in future colliders](https://mlsanity.com/paper/2503.00131/) (2025)
- [Pretrained Event Classification Model for High Energy Physics Analysis](https://mlsanity.com/paper/2412.10665/) (2024)
- [Towards Foundation Models for Experimental Readout Systems Combining Discrete and Continuous Data](https://mlsanity.com/paper/2505.08736/) (2025)
- [Double Descent and Overparameterization in Particle Physics Data](https://mlsanity.com/paper/2509.01397/) (2025)
- [Towards foundation-style models for energy-frontier heterogeneous neutrino detectors via self-supervised pre-training](https://mlsanity.com/paper/2604.07037/) (2026)
- [Are We Ready for AI-Driven Discovery? AI Verification Before the Next Fundamental Physics Breakthrough](https://mlsanity.com/paper/2607.10039/) (2026)
- [Prompting Particle Physics: Tokenized Multi-modal Foundation Models for Combinatorially Many Tasks](https://mlsanity.com/paper/2609.31862/) (2026)
- [Application of a Mixture of Experts-based Foundation Model to the GlueX DIRC Detector](https://mlsanity.com/paper/2604.24775/) (2026)
- [Machine-learned particle flow as a foundation model for collider physics](https://mlsanity.com/paper/2606.14373/) (2026)
- [Communicating Likelihoods with Normalising Flows](https://mlsanity.com/paper/2502.09494/) (2025)
- [Finding and using interpretable latents in a neutrino foundation model with sparse autoencoders](https://mlsanity.com/paper/2608.26090/) (2026)

## Abstract (arXiv)

In this work we demonstrate that significant gains in performance and data efficiency can be achieved in High Energy Physics (HEP) by moving beyond the standard paradigm of sequential optimization or reconstruction and analysis components. We conceptually connect HEP reconstruction and analysis to modern machine learning workflows such as pretraining, finetuning, domain adaptation and high-dimensional embedding spaces and quantify the gains in the example usecase of searches of heavy resonances decaying via an intermediate di-Higgs system to four b-jets.