Topological Data Analysis Applications in Natural Language Processing: A Survey
Not on the map. Why?
Abstract (arXiv)
The surge of data available on the Internet has driven the adoption of a wide range of computational methods for analyzing and extracting insights from large-scale data. Among these, Machine Learning (ML) has become a central paradigm, offering powerful tools for pattern discovery, prediction, and representation learning across many domains. At the same time, real-world data often exhibit properties such as noise, imbalance, sparsity, limited supervision, and high dimensionality, motivating the use of additional analytical perspectives that can complement standard ML pipelines. One such perspective is Topological Data Analysis (TDA), a statistical framework that focuses on the intrinsic shape and structural organization of data. Rather than replacing ML, TDA offers a complementary lens for characterizing geometric and topological properties that may be difficult to capture with conventional feature-based or purely predictive approaches. This has motivated a growing body of work that integrates TDA into ML workflows, particularly in settings where data structure plays an important role. Despite this promise, TDA has received relatively limited attention in Natural Language Processing (NLP) compared to domains with more overt structural regularities, such as computer vision. Nevertheless, a dedicated community of researchers has explored its use in NLP, leading to 137 papers that we comprehensively survey in this work. We organize these studies into theoretical and nontheoretical approaches. Theoretical approaches use topology to explain linguistic phenomena, whereas non-theoretical approaches incorporate TDA into ML-based pipelines through a variety of numerical representations. We conclude by discussing the key challenges and open questions that continue to shape this emerging area. Resources and a list of papers are available at: https://github.com/AdaUchendu/AwesomeTDA4NLP.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 62 of its 178 references that are in the map. First posted on arXiv 15 Nov 2024.
Builds on
62 of 178 references.
References · 62
- 2017Attention Is All You Needcited by 60,361
- 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
- 2018Deep contextualized word representationscited by 2,290
- 2023Code Llama: Open Foundation Models for Codecited by 2,139
- 2016Enriching Word Vectors with Subword Informationcited by 1,583
- 2015Persistence Images: A Stable Vector Representation of Persistent Homologycited by 236
- 2019Ripser: efficient computation of Vietoris-Rips persistence barcodescited by 117
- 2023The geometry of hidden representations of large transformer modelscited by 112
- 2023Intrinsic Dimension Estimation for Robust Detection of AI-Generated Textscited by 112
- 2020Simplicial Neural Networkscited by 110
- 2021Topological Graph Neural Networkscited by 87
- 2024Position: Topological Deep Learning is the New Frontier for Relational Learningcited by 66
38 more
- 2021Artificial Text Detection via Examining the Topology of Attention Mapscited by 49
- 2023Learning Topology-Preserving Data Representationscited by 31
- 2023AI-generated text boundary detection with RoFTcited by 20
- 2022Acceptability Judgements via Examining the Topology of Attention Mapscited by 19
- 2020Topology of Word Embeddings: Singularities Reflect Polysemycited by 15
- 2024Persistent Topological Features in Large Language Modelscited by 12
- 2022The Topological BERT: Transforming Attention into Topology for Natural Language Processingcited by 10
- 2023TOPFORMER: Topology-Aware Authorship Attribution of Deepfake Texts with Diverse Writing Stylescited by 10
- 2019Topological Analysis of Syntactic Structurescited by 9
- 2022Topological Data Analysis for Speech Processingcited by 9
- 2023Uncertainty Estimation of Transformers' Predictions via Topological Analysis of the Attention Matricescited by 8
- 2017Does the Geometry of Word Embeddings Help Document Classification? A Case Study on Persistent Homology Based Representationscited by 6
- 2024Hidden Holes: topological aspects of language modelscited by 6
- 2022BERTops: Studying BERT Representations under a Topological Lenscited by 5
- 2022Dialogue Term Extraction using Transfer Learning and Topological Data Analysiscited by 5
- 2023Can BERT eat RuCoLA? Topological Data Analysis to Explaincited by 5
- 2015Persistent Topology of Syntaxcited by 4
- 2018A Short Survey of Topological Data Analysis in Time Series and Systems Analysiscited by 4
- 2024The Shape of Word Embeddings: Quantifying Non-Isometry With Topological Data Analysiscited by 4
- 2024MOUNTAINEER: Topology-Driven Visual Analytics for Comparing Local Explanationscited by 4
- 2024Relative Representations: Topological and Geometric Perspectivescited by 4
- 2024Functional Homotopy: Smoothing Discrete Optimization via Continuous Parameters for LLM Jailbreak Attackscited by 4
- 2018Towards Emotion Recognition: A Persistent Entropy Applicationcited by 3
- 2019An Introduction to a New Text Classification and Visualization for Natural Language Processing Using Topological Data Analysiscited by 3
- 2019Architecture and evolution of semantic networks in mathematics textscited by 3
- 2020Argumentative Topology: Finding Loop(holes) in Logiccited by 3
- 2021Persistence Homology of TEDtalk: Do Sentence Embeddings Have a Topological Shape?cited by 3
- 2023Do Large GPT Models Discover Moral Dimensions in Language Representations? A Topological Study Of Sentence Embeddingscited by 3
- 2023ChatGPT for Computational Topologycited by 3
- 2024Topological Perspectives on Optimal Multimodal Embedding Spacescited by 3
- 2018Local Homology of Word Embeddingscited by 2
- 2021Abstraction, Reasoning and Deep Learning: A Study of the "Look and Say" Sequencecited by 2
- 2023Topological Interpretability for Deep-Learningcited by 2
- 2023Trainable and Explainable Simplicial Map Neural Networkscited by 2
- 2023Topological Interpretations of GPT-3cited by 2
- 2023Detecting out-of-distribution text using topological features of transformer-based language modelscited by 2
- 2024Local Topology Measures of Contextual Language Model Latent Spaces With Applications to Dialogue Term Extractioncited by 2
- 2019Summary and Distance between Sets of Texts based on Topological Data Analysiscited by 1
50 most cited of 62 All 62 (JSON)
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.