# Multiset Transformer: Advancing Representation Learning in Persistence Diagrams

arXiv: https://arxiv.org/abs/2411.14662 · HTML: https://mlsanity.com/paper/2411.14662/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 41 references that are in the map.
- First posted on arXiv 22 Nov 2024.

## Builds on (most cited first, 5 of 5)

- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Topological Graph Neural Networks](https://mlsanity.com/paper/2102.07835/) (2021)
- [Persformer: A Transformer Architecture for Topological Machine Learning](https://mlsanity.com/paper/2112.15210/) (2021)
- [Multiset-Equivariant Set Prediction with Approximate Implicit Differentiation](https://mlsanity.com/paper/2111.12193/) (2021)
- [Deep Learning with Sets and Point Clouds](https://mlsanity.com/paper/1611.04500/) (2016)

## Built on by (most cited first, 2 of 2)

- [FILTR: Extracting Topological Features from Pretrained 3D Models](https://mlsanity.com/paper/2604.22334/) (2026)
- [Voronoi Histograms for Adaptive Vectorization of Expected Persistence Diagrams](https://mlsanity.com/paper/2607.27126/) (2026)

## Abstract (arXiv)

To improve persistence diagram representation learning, we propose Multiset Transformer. This is the first neural network that utilizes attention mechanisms specifically designed for multisets as inputs and offers rigorous theoretical guarantees of permutation invariance. The architecture integrates multiset-enhanced attentions with a pool-decomposition scheme, allowing multiplicities to be preserved across equivariant layers. This capability enables full leverage of multiplicities while significantly reducing both computational and spatial complexity compared to the Set Transformer. Additionally, our method can greatly benefit from clustering as a preprocessing step to further minimize complexity, an advantage not possessed by the Set Transformer. Experimental results demonstrate that the Multiset Transformer outperforms existing neural network methods in the realm of persistence diagram representation learning.