# SPANet: Generalized Permutationless Set Assignment for Particle Physics using Symmetry Preserving Attention

arXiv: https://arxiv.org/abs/2106.03898 · HTML: https://mlsanity.com/paper/2106.03898/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 57 references that are in the map.
- First posted on arXiv 7 Jun 2021.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [Gauge Equivariant Convolutional Networks and the Icosahedral CNN](https://mlsanity.com/paper/1902.04615/) (2019)
- [Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks](https://mlsanity.com/paper/1810.00825/) (2018)
- [Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs](https://mlsanity.com/paper/2003.05425/) (2020)
- [DeepPermNet: Visual Permutation Learning](https://mlsanity.com/paper/1704.02729/) (2017)
- [Permutationless Many-Jet Event Reconstruction with Symmetry Preserving Attention Networks](https://mlsanity.com/paper/2010.09206/) (2020)
- [Sherpa: Robust Hyperparameter Optimization for Machine Learning](https://mlsanity.com/paper/2005.04048/) (2020)
- [Estimating Cosmological Parameters from the Dark Matter Distribution](https://mlsanity.com/paper/1711.02033/) (2017)
- [End-to-End Jet Classification of Quarks and Gluons with the CMS Open Data](https://mlsanity.com/paper/1902.08276/) (2019)
- [MidRank: Learning to rank based on subsequences](https://mlsanity.com/paper/1511.08951/) (2015)

## Built on by (most cited first, 18 of 18)

- [Deep Learning Symmetries and Their Lie Groups, Algebras, and Subalgebras from First Principles](https://mlsanity.com/paper/2301.05638/) (2023)
- [End-To-End Latent Variational Diffusion Models for Inverse Problems in High Energy Physics](https://mlsanity.com/paper/2305.10399/) (2023)
- [nu-Flows: Conditional Neutrino Regression](https://mlsanity.com/paper/2207.00664/) (2022)
- [Snowmass 2021 Computational Frontier CompF03 Topical Group Report: Machine Learning](https://mlsanity.com/paper/2209.07559/) (2022)
- [nu2-Flows: Fast and improved neutrino reconstruction in multi-neutrino final states with conditional normalizing flows](https://mlsanity.com/paper/2307.02405/) (2023)
- [Topological Reconstruction of Particle Physics Processes using Graph Neural Networks](https://mlsanity.com/paper/2303.13937/) (2023)
- [The Quarks of Attention](https://mlsanity.com/paper/2202.08371/) (2022)
- [Reconstruction of Unstable Heavy Particles Using Deep Symmetry-Preserving Attention Networks](https://mlsanity.com/paper/2309.01886/) (2023)
- [Ultra Fast Transformers on FPGAs for Particle Physics Experiments](https://mlsanity.com/paper/2402.01047/) (2024)
- [Oracle-Preserving Latent Flows](https://mlsanity.com/paper/2302.00806/) (2023)
- [Optimal Equivariant Architectures from the Symmetries of Matrix-Element Likelihoods](https://mlsanity.com/paper/2410.18553/) (2024)
- [Reconstructing short-lived particles using hypergraph representation learning](https://mlsanity.com/paper/2402.10149/) (2024)
- [Interpretable Joint Event-Particle Reconstruction for Neutrino Physics at NOvA with Sparse CNNs and Transformers](https://mlsanity.com/paper/2303.06201/) (2023)
- [Reconstruction of boosted and resolved multi-Higgs-boson events with symmetry-preserving attention networks](https://mlsanity.com/paper/2412.03819/) (2024)
- [Pointwise Metrics Mislead: An Evaluation Protocol for Multimodal Inverse Problems](https://mlsanity.com/paper/2605.22891/) (2026)
- [Pairton: Iterative Reconstruction of Short-Lived Particles](https://mlsanity.com/paper/2608.14278/) (2026)
- [Probing SMEFT Operators through tbarttbart Production with Hyper-Graph Neural Networks at the LHC](https://mlsanity.com/paper/2605.18382/) (2026)
- [ResoSeg: Resonance Tagger using Transformer and Segment Model](https://mlsanity.com/paper/2609.12610/) (2026)

## Abstract (arXiv)

The creation of unstable heavy particles at the Large Hadron Collider is the most direct way to address some of the deepest open questions in physics. Collisions typically produce variable-size sets of observed particles which have inherent ambiguities complicating the assignment of observed particles to the decay products of the heavy particles. Current strategies for tackling these challenges in the physics community ignore the physical symmetries of the decay products and consider all possible assignment permutations and do not scale to complex configurations. Attention based deep learning methods for sequence modelling have achieved state-of-the-art performance in natural language processing, but they lack built-in mechanisms to deal with the unique symmetries found in physical set-assignment problems. We introduce a novel method for constructing symmetry-preserving attention networks which reflect the problem's natural invariances to efficiently find assignments without evaluating all permutations. This general approach is applicable to arbitrarily complex configurations and significantly outperforms current methods, improving reconstruction efficiency between 19% - 35% on typical benchmark problems while decreasing inference time by two to five orders of magnitude on the most complex events, making many important and previously intractable cases tractable. A full code repository containing a general library, the specific configuration used, and a complete dataset release, are avaiable at https://github.com/Alexanders101/SPANet