# Point Cloud Transformers applied to Collider Physics

arXiv: https://arxiv.org/abs/2102.05073 · HTML: https://mlsanity.com/paper/2102.05073/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 33 references that are in the map.
- First posted on arXiv 9 Feb 2021.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [Visual Transformers: Token-based Image Representation and Processing for Computer Vision](https://mlsanity.com/paper/2006.03677/) (2020)
- [Fast inference of deep neural networks in FPGAs for particle physics](https://mlsanity.com/paper/1804.06913/) (2018)
- [Energy Flow Networks: Deep Sets for Particle Jets](https://mlsanity.com/paper/1810.05165/) (2018)
- [ParticleNet: Jet Tagging via Particle Clouds](https://mlsanity.com/paper/1902.08570/) (2019)
- [Learning representations of irregular particle-detector geometry with distance-weighted graph networks](https://mlsanity.com/paper/1902.07987/) (2019)
- [Permutationless Many-Jet Event Reconstruction with Symmetry Preserving Attention Networks](https://mlsanity.com/paper/2010.09206/) (2020)
- [Deep learning in color: towards automated quark/gluon jet discrimination](https://mlsanity.com/paper/1612.01551/) (2016)
- [GAPNet: Graph Attention based Point Neural Network for Exploiting Local Feature of Point Cloud](https://mlsanity.com/paper/1905.08705/) (2019)

## Built on by (most cited first, 23 of 23)

- [Particle Transformer for Jet Tagging](https://mlsanity.com/paper/2202.03772/) (2022)
- [Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Models](https://mlsanity.com/paper/2401.13537/) (2024)
- [PC-JeDi: Diffusion for Particle Cloud Generation in High Energy Physics](https://mlsanity.com/paper/2303.05376/) (2023)
- [Explainable Equivariant Neural Networks for Particle Physics: PELICAN](https://mlsanity.com/paper/2307.16506/) (2023)
- [Snowmass 2021 Computational Frontier CompF03 Topical Group Report: Machine Learning](https://mlsanity.com/paper/2209.07559/) (2022)
- [Polynomial Width is Sufficient for Set Representation with High-dimensional Features](https://mlsanity.com/paper/2307.04001/) (2023)
- [A Comparison Between Invariant and Equivariant Classical and Quantum Graph Neural Networks](https://mlsanity.com/paper/2311.18672/) (2023)
- [Ultra Fast Transformers on FPGAs for Particle Physics Experiments](https://mlsanity.com/paper/2402.01047/) (2024)
- [Learning the Simplicity of Scattering Amplitudes](https://mlsanity.com/paper/2408.04720/) (2024)
- [TrackFormers: In Search of Transformer-Based Particle Tracking for the High-Luminosity LHC Era](https://mlsanity.com/paper/2407.07179/) (2024)
- [Feature Selection with Distance Correlation](https://mlsanity.com/paper/2212.00046/) (2022)
- [PCN: A Deep Learning Approach to Jet Tagging Utilizing Novel Graph Construction Methods and Chebyshev Graph Convolutions](https://mlsanity.com/paper/2309.08630/) (2023)
- [Vision Transformers for End-to-End Quark-Gluon Jet Classification from Calorimeter Images](https://mlsanity.com/paper/2506.14934/) (2025)
- [Stable and Interpretable Jet Physics with IRC-Safe Equivariant Feature Extraction](https://mlsanity.com/paper/2509.22059/) (2025)
- [Prompting Particle Physics: Tokenized Multi-modal Foundation Models for Combinatorially Many Tasks](https://mlsanity.com/paper/2609.31862/) (2026)
- [Image and Point-cloud Classification for Jet Analysis in High-Energy Physics: A survey](https://mlsanity.com/paper/2403.11934/) (2024)
- [Generating particle physics Lagrangians with transformers](https://mlsanity.com/paper/2501.09729/) (2025)
- [BitHEP -- The Limits of Low-Precision ML in HEP](https://mlsanity.com/paper/2504.03387/) (2025)
- [KIGNet: Physics-Motivated Multi-Graph Representation Learning for Explainable Jet Tagging](https://mlsanity.com/paper/2512.07420/) (2025)
- [From Small to Large: Generalization Bounds for Transformers on Variable-Size Inputs](https://mlsanity.com/paper/2512.12805/) (2025)
- [Deep learning techniques for energy clustering in the CMS ECAL](https://mlsanity.com/paper/2204.10277/) (2022)
- [POViT: Vision Transformer for Multi-objective Design and Characterization of Nanophotonic Devices](https://mlsanity.com/paper/2205.09045/) (2022)
- [Transfer Learning Across Fast- and Full-Simulation Domains in High-Energy Physics](https://mlsanity.com/paper/2605.07471/) (2026)

## Abstract (arXiv)

Methods for processing point cloud information have seen a great success in collider physics applications. One recent breakthrough in machine learning is the usage of Transformer networks to learn semantic relationships between sequences in language processing. In this work, we apply a modified Transformer network called Point Cloud Transformer as a method to incorporate the advantages of the Transformer architecture to an unordered set of particles resulting from collision events. To compare the performance with other strategies, we study jet-tagging applications for highly-boosted particles.