# Persformer: A Transformer Architecture for Topological Machine Learning

arXiv: https://arxiv.org/abs/2112.15210 · HTML: https://mlsanity.com/paper/2112.15210/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 41 references that are in the map.
- First posted on arXiv 30 Dec 2021.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Scaling Vision Transformers](https://mlsanity.com/paper/2106.04560/) (2021)
- [Persistence Images: A Stable Vector Representation of Persistent Homology](https://mlsanity.com/paper/1507.06217/) (2015)
- [PersLay: A Neural Network Layer for Persistence Diagrams and New Graph Topological Signatures](https://mlsanity.com/paper/1904.09378/) (2019)
- [Deep Learning with Topological Signatures](https://mlsanity.com/paper/1707.04041/) (2017)
- [Topology of deep neural networks](https://mlsanity.com/paper/2004.06093/) (2020)
- [Training Tips for the Transformer Model](https://mlsanity.com/paper/1804.00247/) (2018)
- [Sliced Wasserstein Kernel for Persistence Diagrams](https://mlsanity.com/paper/1706.03358/) (2017)
- [A Stable Multi-Scale Kernel for Topological Machine Learning](https://mlsanity.com/paper/1412.6821/) (2014)
- [giotto-tda: A Topological Data Analysis Toolkit for Machine Learning and Data Exploration](https://mlsanity.com/paper/2004.02551/) (2020)
- [Persistence Fisher Kernel: A Riemannian Manifold Kernel for Persistence Diagrams](https://mlsanity.com/paper/1802.03569/) (2018)
- [Persistent homology detects curvature](https://mlsanity.com/paper/1905.13196/) (2019)
- [Kernel method for persistence diagrams via kernel embedding and weight factor](https://mlsanity.com/paper/1706.03472/) (2017)
- [Topological Uncertainty: Monitoring trained neural networks through persistence of activation graphs](https://mlsanity.com/paper/2105.04404/) (2021)
- [Topological analysis of the connectome of digital reconstructions of neural microcircuits](https://mlsanity.com/paper/1601.01580/) (2016)
- [Visualising the Evolution of English Covid-19 Cases with Topological Data Analysis Ball Mapper](https://mlsanity.com/paper/2004.03282/) (2020)

## Built on by (most cited first, 19 of 19)

- [Topological Deep Learning: A Review of an Emerging Paradigm](https://mlsanity.com/paper/2302.03836/) (2023)
- [Euler Characteristic Tools For Topological Data Analysis](https://mlsanity.com/paper/2303.14040/) (2023)
- [The Topological BERT: Transforming Attention into Topology for Natural Language Processing](https://mlsanity.com/paper/2206.15195/) (2022)
- [Topology-Aware 3D Gaussian Splatting: Leveraging Persistent Homology for Optimized Structural Integrity](https://mlsanity.com/paper/2412.16619/) (2024)
- [Dialogue Term Extraction using Transfer Learning and Topological Data Analysis](https://mlsanity.com/paper/2208.10448/) (2022)
- [Topological Dictionary Learning](https://mlsanity.com/paper/2503.11470/) (2025)
- [SparseMeXT Unlocking the Potential of Sparse Representations for HD Map Construction](https://mlsanity.com/paper/2505.08808/) (2025)
- [A Closed-Form Persistence-Landmark Pipeline for Certified Point-Cloud and Graph Classification](https://mlsanity.com/paper/2605.02836/) (2026)
- [Statistical learning on measures: an application to persistence diagrams](https://mlsanity.com/paper/2303.08456/) (2023)
- [A Closed-Form Adaptive-Landmark Kernel for Certified Point-Cloud and Graph Classification](https://mlsanity.com/paper/2605.04046/) (2026)
- [Multiset Transformer: Advancing Representation Learning in Persistence Diagrams](https://mlsanity.com/paper/2411.14662/) (2024)
- [nD-RoPE: A Generalized RoPE for n-Dimensional Position Embedding](https://mlsanity.com/paper/2606.12146/) (2026)
- [Discrete transforms of quantized persistence diagrams](https://mlsanity.com/paper/2312.17093/) (2023)
- [Decorrelating neurons using persistence](https://mlsanity.com/paper/2308.04870/) (2023)
- [GridPE: A Grid Cell-Inspired Unified Position Embedding for Arbitrary-Dimensional Spaces](https://mlsanity.com/paper/2406.07049/) (2024)
- [xPerT: Extended Persistence Transformer](https://mlsanity.com/paper/2410.14193/) (2024)
- [TUN: Detecting Significant Points in Persistence Diagrams with Deep Learning](https://mlsanity.com/paper/2512.14274/) (2025)
- [COMPLEX: A Closed-Form Certified Embedding of Multiparameter Persistence Modules](https://mlsanity.com/paper/2609.22012/) (2026)
- [LandscapeSHAP: Which Persistent Homology Class Gets the Credit?](https://mlsanity.com/paper/2609.31469/) (2026)

## Abstract (arXiv)

One of the main challenges of Topological Data Analysis (TDA) is to extract features from persistent diagrams directly usable by machine learning algorithms. Indeed, persistence diagrams are intrinsically (multi-)sets of points in ℝ2 and cannot be seen in a straightforward manner as vectors. In this article, we introduce textttPersformer, the first Transformer neural network architecture that accepts persistence diagrams as input. The textttPersformer architecture significantly outperforms previous topological neural network architectures on classical synthetic and graph benchmark datasets. Moreover, it satisfies a universal approximation theorem. This allows us to introduce the first interpretability method for topological machine learning, which we explore in two examples.