# TrackFormers Part 2: Enhanced Transformer-Based Models for High-Energy Physics Track Reconstruction

arXiv: https://arxiv.org/abs/2509.26411 · HTML: https://mlsanity.com/paper/2509.26411/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 3 of its 9 references that are in the map.
- First posted on arXiv 30 Sep 2025.

## Builds on (most cited first, 3 of 3)

- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Flex Attention: A Programming Model for Generating Optimized Attention Kernels](https://mlsanity.com/paper/2412.05496/) (2024)
- [TrackFormers: In Search of Transformer-Based Particle Tracking for the High-Luminosity LHC Era](https://mlsanity.com/paper/2407.07179/) (2024)

## Abstract (arXiv)

High-Energy Physics experiments are rapidly escalating in generated data volume, a trend that will intensify with the upcoming High-Luminosity LHC upgrade. This surge in data necessitates critical revisions across the data processing pipeline, with particle track reconstruction being a prime candidate for improvement. In our previous work, we introduced "TrackFormers", a collection of Transformer-based one-shot encoder-only models that effectively associate hits with expected tracks. In this study, we extend our earlier efforts by conducting detailed investigations into more custom Transformer attention mechanisms, a new design combining geometric projection and lightweight clustering, and a joint model conditioning classification on a regressor's predictions. Furthermore, we discuss new datasets that allow the training on hit level for a range of physics processes. These developments collectively aim to boost both the accuracy and potentially the efficiency of our tracking models, offering a robust solution to meet the demands of next-generation high-energy physics experiments.