mlsanityData to 14 Oct 2026

Finetuning Foundation Models for Joint Analysis Optimization

arXiv 2401.13536 ↗First posted 24 Jan 2024hep-ex cs.LG hep-phcited by 18

Not on the map. Why?

Abstract (arXiv)

In this work we demonstrate that significant gains in performance and data efficiency can be achieved in High Energy Physics (HEP) by moving beyond the standard paradigm of sequential optimization or reconstruction and analysis components. We conceptually connect HEP reconstruction and analysis to modern machine learning workflows such as pretraining, finetuning, domain adaptation and high-dimensional embedding spaces and quantify the gains in the example usecase of searches of heavy resonances decaying via an intermediate di-Higgs system to four b-jets.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 31 of its 62 references that are in the map. First posted on arXiv 24 Jan 2024.

Builds on

31 of 62 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2023GPT-4 Technical Reportcited by 18,396
  6. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  8. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  9. 2017Deep Setscited by 1,421
  10. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  11. 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 772
  12. 2019The frontier of simulation-based inferencecited by 502
19 more
  1. 2019Lookahead Optimizer: k steps forward, 1 step backcited by 336
  2. 2023HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolutioncited by 194
  3. 2022Particle Transformer for Jet Taggingcited by 98
  4. 2018Energy Flow Networks: Deep Sets for Particle Jetscited by 84
  5. 2021MLPF: Efficient machine-learned particle-flow reconstruction using graph neural networkscited by 36
  6. 2021Performance of a Geometric Deep Learning Pipeline for HL-LHC Particle Trackingcited by 30
  7. 2023Disentangling Neuron Representations with Concept Vectorscited by 28
  8. 2017QCD-Aware Recursive Neural Networks for Jet Physicscited by 22
  9. 2021Charged particle tracking via edge-classifying interaction networkscited by 19
  10. 2018INFERNO: Inference-Aware Neural Optimisationcited by 16
  11. 2021A FAIR and AI-ready Higgs boson decay datasetcited by 14
  12. 2022Machine Learning for Particle Flow Reconstruction at CMScited by 14
  13. 2022neos: End-to-End-Optimised Summary Statistics for High Energy Physicscited by 10
  14. 2022Differentiable Matrix Elements with MadJaxcited by 8
  15. 2023Hierarchical Graph Neural Networks for Particle Track Reconstructioncited by 7
  16. 2023Towards a Foundation Model for Neural Network Wavefunctionscited by 7
  17. 2023Branches of a Tree: Taking Derivatives of Programs with Discrete and Branching Randomness in High Energy Physicscited by 6
  18. 2022Leveraging universality of jet taggers through transfer learningcited by 5
  19. 2023Differentiable Vertex Fitting for Jet Flavour Taggingcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 6; 2026: 7.

18 papers.

Sort:

By area:

  1. 2024OmniJet-α: The first cross-task foundation model for particle physicscited by 43
  2. 2024Re-Simulation-based Self-Supervised Learning for Pre-Training Foundation Modelscited by 22
  3. 2024Is Tokenization Needed for Masked Particle Modelling?cited by 22
  4. 2024PAPERCLIP: Associating Astronomical Observations and Natural Language with Multi-Modal Modelscited by 15
  5. 2025Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Modelscited by 14
  6. 2026Neural Scaling Laws for Boosted Jet Taggingcited by 12
  7. 2025Foundation models for high-energy physicscited by 10
  8. 2025Fine-tuning machine-learned particle-flow reconstruction for new detector geometries in future colliderscited by 9
  9. 2024Pretrained Event Classification Model for High Energy Physics Analysiscited by 8
  10. 2025Towards Foundation Models for Experimental Readout Systems Combining Discrete and Continuous Datacited by 5
  11. 2025Double Descent and Overparameterization in Particle Physics Datacited by 2
  12. 2026Towards foundation-style models for energy-frontier heterogeneous neutrino detectors via self-supervised pre-trainingcited by 2
6 more
  1. 2026Are We Ready for AI-Driven Discovery? AI Verification Before the Next Fundamental Physics Breakthroughcited by 2
  2. 2026Prompting Particle Physics: Tokenized Multi-modal Foundation Models for Combinatorially Many Taskscited by 2
  3. 2026Application of a Mixture of Experts-based Foundation Model to the GlueX DIRC Detectorcited by 1
  4. 2026Machine-learned particle flow as a foundation model for collider physicscited by 1
  5. 2025Communicating Likelihoods with Normalising Flowscited by 0
  6. 2026Finding and using interpretable latents in a neutrino foundation model with sparse autoencoderscited by 0