# Particle Multi-Axis Transformer for Jet Tagging

arXiv: https://arxiv.org/abs/2406.06638 · HTML: https://mlsanity.com/paper/2406.06638/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 42 references that are in the map.
- First posted on arXiv 9 Jun 2024.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Going deeper with Image Transformers](https://mlsanity.com/paper/2103.17239/) (2021)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [MaxViT: Multi-Axis Vision Transformer](https://mlsanity.com/paper/2204.01697/) (2022)
- [Particle Transformer for Jet Tagging](https://mlsanity.com/paper/2202.03772/) (2022)
- [Energy Flow Networks: Deep Sets for Particle Jets](https://mlsanity.com/paper/1810.05165/) (2018)
- [Particle Cloud Generation with Message Passing Generative Adversarial Networks](https://mlsanity.com/paper/2106.11535/) (2021)
- [Jet-Images -- Deep Learning Edition](https://mlsanity.com/paper/1511.05190/) (2015)
- [A survey of the Vision Transformers and their CNN-Transformer based Variants](https://mlsanity.com/paper/2305.09880/) (2023)
- [A Recent Survey of Vision Transformers for Medical Image Segmentation](https://mlsanity.com/paper/2312.00634/) (2023)
- [Jet tagging in the Lund plane with graph networks](https://mlsanity.com/paper/2012.08526/) (2020)
- [IoT Malware Detection Architecture using a Novel Channel Boosted and Squeezed CNN](https://mlsanity.com/paper/2202.04121/) (2022)
- [MaxViT-UNet: Multi-Axis Attention for Medical Image Segmentation](https://mlsanity.com/paper/2305.08396/) (2023)
- [Particle identification with machine learning from incomplete data in the ALICE experiment](https://mlsanity.com/paper/2403.17436/) (2024)
- [RSIR Transformer: Hierarchical Vision Transformer using Random Sampling Windows and Important Region Windows](https://mlsanity.com/paper/2304.06250/) (2023)

## Abstract (arXiv)

Jet tagging is an essential categorization problem in high energy physics. In recent times, Deep Learning has not only risen to the challenge of jet tagging but also significantly improved its performance. In this article, we proposed an idea of a new architecture, Particle Multi-Axis transformer (ParMAT) which is a modified version of Particle transformer (ParT). ParMAT contains local and global spatial interactions within a single unit which improves its ability to handle various input lengths. We trained our model on JETCLASS, a publicly available large dataset that contains 100M jets of 10 different classes of particles. By integrating a parallel attention mechanism and pairwise interactions of particles in the attention mechanism, ParMAT achieves robustness and higher accuracy over the ParT and ParticleNet. The scalability of the model to huge datasets and its ability to automatically extract essential features demonstrate its potential for enhancing jet tagging.