# Particle Transformer for Jet Tagging

arXiv: https://arxiv.org/abs/2202.03772 · HTML: https://mlsanity.com/paper/2202.03772/

## Key facts

- Cited by 98 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 68 references that are in the map.
- First posted on arXiv 8 Feb 2022.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [Going deeper with Image Transformers](https://mlsanity.com/paper/2103.17239/) (2021)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [Lookahead Optimizer: k steps forward, 1 step back](https://mlsanity.com/paper/1907.08610/) (2019)
- [Lorentz Group Equivariant Neural Network for Particle Physics](https://mlsanity.com/paper/2006.04780/) (2020)
- [Energy Flow Networks: Deep Sets for Particle Jets](https://mlsanity.com/paper/1810.05165/) (2018)
- [NormFormer: Improved Transformer Pretraining with Extra Normalization](https://mlsanity.com/paper/2110.09456/) (2021)
- [ParticleNet: Jet Tagging via Particle Clouds](https://mlsanity.com/paper/1902.08570/) (2019)
- [Particle Cloud Generation with Message Passing Generative Adversarial Networks](https://mlsanity.com/paper/2106.11535/) (2021)
- [Jet-Images -- Deep Learning Edition](https://mlsanity.com/paper/1511.05190/) (2015)
- [Point Cloud Transformers applied to Collider Physics](https://mlsanity.com/paper/2102.05073/) (2021)
- [QCD-Aware Recursive Neural Networks for Jet Physics](https://mlsanity.com/paper/1702.00748/) (2017)
- [Jet tagging in the Lund plane with graph networks](https://mlsanity.com/paper/2012.08526/) (2020)
- [GAPNet: Graph Attention based Point Neural Network for Exploiting Local Feature of Point Cloud](https://mlsanity.com/paper/1905.08705/) (2019)
- [A FAIR and AI-ready Higgs boson decay dataset](https://mlsanity.com/paper/2108.02214/) (2021)
- [Particle Convolution for High Energy Physics](https://mlsanity.com/paper/2107.02908/) (2021)

## Built on by (most cited first, 30 of 98)

- [OmniJet-α: The first cross-task foundation model for particle physics](https://mlsanity.com/paper/2403.05618/) (2024)
- [Lorentz-Equivariant Geometric Algebra Transformers for High-Energy Physics](https://mlsanity.com/paper/2405.14806/) (2024)
- [A Lorentz-Equivariant Transformer for All of the LHC](https://mlsanity.com/paper/2411.00446/) (2024)
- [PC-JeDi: Diffusion for Particle Cloud Generation in High Energy Physics](https://mlsanity.com/paper/2303.05376/) (2023)
- [Explainable Equivariant Neural Networks for Particle Physics: PELICAN](https://mlsanity.com/paper/2307.16506/) (2023)
- [Is Tokenization Needed for Masked Particle Modelling?](https://mlsanity.com/paper/2409.12589/) (2024)
- [Deep Generative Models for Detector Signature Simulation: A Taxonomic Review](https://mlsanity.com/paper/2312.09597/) (2023)
- [Finetuning Foundation Models for Joint Analysis Optimization](https://mlsanity.com/paper/2401.13536/) (2024)
- [A General Framework for Equivariant Neural Networks on Reductive Lie Groups](https://mlsanity.com/paper/2306.00091/) (2023)
- [OmniJet-αC: Learning point cloud calorimeter simulations using generative transformers](https://mlsanity.com/paper/2501.05534/) (2025)
- [Ultrafast jet classification on FPGAs for the HL-LHC](https://mlsanity.com/paper/2402.01876/) (2024)
- [Locality-Sensitive Hashing-Based Efficient Point Transformer with Applications in High-Energy Physics](https://mlsanity.com/paper/2402.12535/) (2024)
- [Neural Scaling Laws for Boosted Jet Tagging](https://mlsanity.com/paper/2602.15781/) (2026)
- [Anomalies, Representations, and Self-Supervision](https://mlsanity.com/paper/2301.04660/) (2023)
- [A Language Model for Particle Tracking](https://mlsanity.com/paper/2402.10239/) (2024)
- [Foundation models for high-energy physics](https://mlsanity.com/paper/2509.21434/) (2025)
- [Fine-tuning machine-learned particle-flow reconstruction for new detector geometries in future colliders](https://mlsanity.com/paper/2503.00131/) (2025)
- [BIP: Boost Invariant Polynomials for Efficient Jet Tagging](https://mlsanity.com/paper/2207.08272/) (2022)
- [A Detailed Study of Interpretability of Deep Neural Network based Top Taggers](https://mlsanity.com/paper/2210.04371/) (2022)
- [Reconstruction of Unstable Heavy Particles Using Deep Symmetry-Preserving Attention Networks](https://mlsanity.com/paper/2309.01886/) (2023)
- [Flow Matching Beyond Kinematics: Generating Jets with Particle-ID and Trajectory Displacement Information](https://mlsanity.com/paper/2312.00123/) (2023)
- [Fast Particle-based Anomaly Detection Algorithm with Variational Autoencoder](https://mlsanity.com/paper/2311.17162/) (2023)
- [Ultra Fast Transformers on FPGAs for Particle Physics Experiments](https://mlsanity.com/paper/2402.01047/) (2024)
- [RINO: Renormalization Group Invariance with No Labels](https://mlsanity.com/paper/2509.07486/) (2025)
- [Sub-microsecond Transformers for Jet Tagging on FPGAs](https://mlsanity.com/paper/2510.24784/) (2025)
- [Leveraging universality of jet taggers through transfer learning](https://mlsanity.com/paper/2203.06210/) (2022)
- [Equivariance Is Not All You Need: Characterizing the Utility of Equivariant Graph Neural Networks for Particle Physics Tasks](https://mlsanity.com/paper/2311.03094/) (2023)
- [Optimal Equivariant Architectures from the Symmetries of Matrix-Element Likelihoods](https://mlsanity.com/paper/2410.18553/) (2024)
- [TrackFormers: In Search of Transformer-Based Particle Tracking for the High-Luminosity LHC Era](https://mlsanity.com/paper/2407.07179/) (2024)
- [Equivariant Neural Networks for General Linear Symmetries on Lie Algebras](https://mlsanity.com/paper/2510.22984/) (2025)

## Abstract (arXiv)

Jet tagging is a critical yet challenging classification task in particle physics. While deep learning has transformed jet tagging and significantly improved performance, the lack of a large-scale public dataset impedes further enhancement. In this work, we present JetClass, a new comprehensive dataset for jet tagging. The JetClass dataset consists of 100 M jets, about two orders of magnitude larger than existing public datasets. A total of 10 types of jets are simulated, including several types unexplored for tagging so far. Based on the large dataset, we propose a new Transformer-based architecture for jet tagging, called Particle Transformer (ParT). By incorporating pairwise particle interactions in the attention mechanism, ParT achieves higher tagging performance than a plain Transformer and surpasses the previous state-of-the-art, ParticleNet, by a large margin. The pre-trained ParT models, once fine-tuned, also substantially enhance the performance on two widely adopted jet tagging benchmarks. The dataset, code and models are publicly available at https://github.com/jet-universe/particle_transformer.