mlsanityData to 7 Oct 2026

A Language Model for Particle Tracking

arXiv 2402.10239 ↗First posted 14 Feb 2024hep-ph cs.LG hep-excited by 10

Not on the map. Why?

Abstract (arXiv)

Particle tracking is crucial for almost all physics analysis programs at the Large Hadron Collider. Deep learning models are pervasively used in particle tracking related tasks. However, the current practice is to design and train one deep learning model for one task with supervised learning techniques. The trained models work well for tasks they are trained on but show no or little generalization capabilities. We propose to unify these models with a language model. In this paper, we present a tokenized detector representation that allows us to train a BERT model for particle tracking. The trained BERT model, namely TrackingBERT, offers latent detector module embedding that can be used for other tasks. This work represents the first step towards developing a foundational model for particle detector understanding.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 10 of its 20 references that are in the map. First posted on arXiv 14 Feb 2024.

Builds on

10 of 20 references.

References · 10

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020Scaling Laws for Neural Language Modelscited by 6,411
  4. 2022Particle Transformer for Jet Taggingcited by 95
  5. 2022Impact of Tokenization on Language Models: An Analysis for Turkishcited by 62
  6. 2021MLPF: Efficient machine-learned particle-flow reconstruction using graph neural networkscited by 36
  7. 2021Performance of a Geometric Deep Learning Pipeline for HL-LHC Particle Trackingcited by 29
  8. 2020Towards a Computer Vision Particle Flowcited by 15
  9. 2022Machine Learning for Particle Flow Reconstruction at CMScited by 14
  10. 2021The Tracking Machine Learning challenge : Throughput phasecited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 3; 2026: 2.

10 papers.

  1. 2024OmniJet-α: The first cross-task foundation model for particle physicscited by 41
  2. 2024Bumblebee: Foundation Model for Particle Physics Discoverycited by 12
  3. 2025Automating High Energy Physics Data Analysis with LLM-Powered Agentscited by 10
  4. 2024Pretrained Event Classification Model for High Energy Physics Analysiscited by 8
  5. 2025Towards Foundation Models for Experimental Readout Systems Combining Discrete and Continuous Datacited by 5
  6. 2024TrackFormers: In Search of Transformer-Based Particle Tracking for the High-Luminosity LHC Eracited by 4
  7. 2025ColliderML: The First Release of an OpenDataDetector High-Luminosity Physics Benchmark Datasetcited by 3
  8. 2026Prompting Particle Physics: Tokenized Multi-modal Foundation Models for Combinatorially Many Taskscited by 1
  9. 2024TrackSorter: A Transformer-based sorting algorithm for track finding in High Energy Physicscited by 0
  10. 2026Learning Standard Model structure from LHC data with Riemannian flow matchingcited by 0