# Topological Data Analysis Applications in Natural Language Processing: A Survey

arXiv: https://arxiv.org/abs/2411.10298 · HTML: https://mlsanity.com/paper/2411.10298/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 62 of its 178 references that are in the map.
- First posted on arXiv 15 Nov 2024.

## Builds on (most cited first, 30 of 62)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Persistence Images: A Stable Vector Representation of Persistent Homology](https://mlsanity.com/paper/1507.06217/) (2015)
- [Ripser: efficient computation of Vietoris-Rips persistence barcodes](https://mlsanity.com/paper/1908.02518/) (2019)
- [The geometry of hidden representations of large transformer models](https://mlsanity.com/paper/2302.00294/) (2023)
- [Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts](https://mlsanity.com/paper/2306.04723/) (2023)
- [Simplicial Neural Networks](https://mlsanity.com/paper/2010.03633/) (2020)
- [Topological Graph Neural Networks](https://mlsanity.com/paper/2102.07835/) (2021)
- [Position: Topological Deep Learning is the New Frontier for Relational Learning](https://mlsanity.com/paper/2402.08871/) (2024)
- [Artificial Text Detection via Examining the Topology of Attention Maps](https://mlsanity.com/paper/2109.04825/) (2021)
- [Learning Topology-Preserving Data Representations](https://mlsanity.com/paper/2302.00136/) (2023)
- [AI-generated text boundary detection with RoFT](https://mlsanity.com/paper/2311.08349/) (2023)
- [Acceptability Judgements via Examining the Topology of Attention Maps](https://mlsanity.com/paper/2205.09630/) (2022)
- [Topology of Word Embeddings: Singularities Reflect Polysemy](https://mlsanity.com/paper/2011.09413/) (2020)
- [Persistent Topological Features in Large Language Models](https://mlsanity.com/paper/2410.11042/) (2024)
- [The Topological BERT: Transforming Attention into Topology for Natural Language Processing](https://mlsanity.com/paper/2206.15195/) (2022)
- [TOPFORMER: Topology-Aware Authorship Attribution of Deepfake Texts with Diverse Writing Styles](https://mlsanity.com/paper/2309.12934/) (2023)
- [Topological Analysis of Syntactic Structures](https://mlsanity.com/paper/1903.05181/) (2019)
- [Topological Data Analysis for Speech Processing](https://mlsanity.com/paper/2211.17223/) (2022)
- [Uncertainty Estimation of Transformers' Predictions via Topological Analysis of the Attention Matrices](https://mlsanity.com/paper/2308.11295/) (2023)
- [Does the Geometry of Word Embeddings Help Document Classification? A Case Study on Persistent Homology Based Representations](https://mlsanity.com/paper/1705.10900/) (2017)
- [Hidden Holes: topological aspects of language models](https://mlsanity.com/paper/2406.05798/) (2024)
- [BERTops: Studying BERT Representations under a Topological Lens](https://mlsanity.com/paper/2205.00953/) (2022)
- [Dialogue Term Extraction using Transfer Learning and Topological Data Analysis](https://mlsanity.com/paper/2208.10448/) (2022)
- [Can BERT eat RuCoLA? Topological Data Analysis to Explain](https://mlsanity.com/paper/2304.01680/) (2023)
- [Persistent Topology of Syntax](https://mlsanity.com/paper/1507.05134/) (2015)
- [A Short Survey of Topological Data Analysis in Time Series and Systems Analysis](https://mlsanity.com/paper/1809.10745/) (2018)

## Built on by (most cited first, 2 of 2)

- [Not Every AI Problem is a Data Problem: We Should Be Intentional About Data Scaling](https://mlsanity.com/paper/2501.13779/) (2025)
- [Topological Signatures of Grokking](https://mlsanity.com/paper/2605.06352/) (2026)

## Abstract (arXiv)

The surge of data available on the Internet has driven the adoption of a wide range of computational methods for analyzing and extracting insights from large-scale data. Among these, Machine Learning (ML) has become a central paradigm, offering powerful tools for pattern discovery, prediction, and representation learning across many domains. At the same time, real-world data often exhibit properties such as noise, imbalance, sparsity, limited supervision, and high dimensionality, motivating the use of additional analytical perspectives that can complement standard ML pipelines. One such perspective is Topological Data Analysis (TDA), a statistical framework that focuses on the intrinsic shape and structural organization of data. Rather than replacing ML, TDA offers a complementary lens for characterizing geometric and topological properties that may be difficult to capture with conventional feature-based or purely predictive approaches. This has motivated a growing body of work that integrates TDA into ML workflows, particularly in settings where data structure plays an important role. Despite this promise, TDA has received relatively limited attention in Natural Language Processing (NLP) compared to domains with more overt structural regularities, such as computer vision. Nevertheless, a dedicated community of researchers has explored its use in NLP, leading to 137 papers that we comprehensively survey in this work. We organize these studies into theoretical and nontheoretical approaches. Theoretical approaches use topology to explain linguistic phenomena, whereas non-theoretical approaches incorporate TDA into ML-based pipelines through a variety of numerical representations. We conclude by discussing the key challenges and open questions that continue to shape this emerging area. Resources and a list of papers are available at: https://github.com/AdaUchendu/AwesomeTDA4NLP.