# Top2Vec: Distributed Representations of Topics

arXiv: https://arxiv.org/abs/2008.09470 · HTML: https://mlsanity.com/paper/2008.09470/

## Key facts

- Cited by 126 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Aug 2020.

## Built on by (most cited first, 30 of 126)

- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [FASTopic: Pretrained Transformer is a Fast, Adaptive, Stable, and Transferable Topic Model](https://mlsanity.com/paper/2405.17978/) (2024)
- [PyThaiNLP: Thai Natural Language Processing in Python](https://mlsanity.com/paper/2312.04649/) (2023)
- [The Scandinavian Embedding Benchmarks: Comprehensive Assessment of Multilingual and Monolingual Text Embedding](https://mlsanity.com/paper/2406.02396/) (2024)
- [Twitter Topic Classification](https://mlsanity.com/paper/2209.09824/) (2022)
- [Contextualized Topic Coherence Metrics](https://mlsanity.com/paper/2305.14587/) (2023)
- [A Survey on Neural Topic Models: Methods, Applications, and Challenges](https://mlsanity.com/paper/2401.15351/) (2024)
- [Tsetlin Machine Embedding: Representing Words Using Logical Expressions](https://mlsanity.com/paper/2301.00709/) (2023)
- [Machine-assisted quantitizing designs: augmenting humanities and social sciences with artificial intelligence](https://mlsanity.com/paper/2309.14379/) (2023)
- [Calibration of Transformer-based Models for Identifying Stress and Depression in Social Media](https://mlsanity.com/paper/2305.16797/) (2023)
- [Happenstance: Utilizing Semantic Search to Track Russian State Media Narratives about the Russo-Ukrainian War On Reddit](https://mlsanity.com/paper/2205.14484/) (2022)
- [Length-Induced Embedding Collapse in PLM-based Models](https://mlsanity.com/paper/2410.24200/) (2024)
- [Semantic-Driven Topic Modeling Using Transformer-Based Embeddings and Clustering Algorithms](https://mlsanity.com/paper/2410.00134/) (2024)
- [Granite Embedding Models](https://mlsanity.com/paper/2502.20204/) (2025)
- [Model Directions, Not Words: Mechanistic Topic Models Using Sparse Autoencoders](https://mlsanity.com/paper/2507.23220/) (2025)
- [Granite Embedding R2 Models](https://mlsanity.com/paper/2508.21085/) (2025)
- [Lbl2Vec: An Embedding-Based Approach for Unsupervised Document Retrieval on Predefined Topics](https://mlsanity.com/paper/2210.06023/) (2022)
- [Specious Sites: Tracking the Spread and Sway of Spurious News Stories at Scale](https://mlsanity.com/paper/2308.02068/) (2023)
- [GPTopic: Dynamic and Interactive Topic Representations](https://mlsanity.com/paper/2403.03628/) (2024)
- [Introducing an Abusive Language Classification Framework for Telegram to Investigate the German Hater Community](https://mlsanity.com/paper/2109.07346/) (2021)
- [Artificial intelligence for topic modelling in Hindu philosophy: mapping themes between the Upanishads and the Bhagavad Gita](https://mlsanity.com/paper/2205.11020/) (2022)
- [ANTM: An Aligned Neural Topic Model for Exploring Evolving Topics](https://mlsanity.com/paper/2302.01501/) (2023)
- [Evolving linguistic divergence on polarizing social media](https://mlsanity.com/paper/2309.01659/) (2023)
- [Computational thematics: Comparing algorithms for clustering the genres of literary fiction](https://mlsanity.com/paper/2305.11251/) (2023)
- [Labeling Free-text Data using Language Model Ensembles](https://mlsanity.com/paper/2501.08413/) (2025)
- [Hatemongers ride on echo chambers to escalate hate speech diffusion](https://mlsanity.com/paper/2302.02479/) (2023)
- [Deep learning for COVID-19 topic modelling via Twitter: Alpha, Delta and Omicron](https://mlsanity.com/paper/2303.00135/) (2023)
- [Topic-Guided Sampling For Data-Efficient Multi-Domain Stance Detection](https://mlsanity.com/paper/2306.00765/) (2023)
- [SEADialogues: A Multilingual Culturally Grounded Multi-turn Dialogue Dataset on Southeast Asian Languages](https://mlsanity.com/paper/2508.07069/) (2025)

## Abstract (arXiv)

Topic modeling is used for discovering latent semantic structure, usually referred to as topics, in a large collection of documents. The most widely used methods are Latent Dirichlet Allocation and Probabilistic Latent Semantic Analysis. Despite their popularity they have several weaknesses. In order to achieve optimal results they often require the number of topics to be known, custom stop-word lists, stemming, and lemmatization. Additionally these methods rely on bag-of-words representation of documents which ignore the ordering and semantics of words. Distributed representations of documents and words have gained popularity due to their ability to capture semantics of words and documents. We present texttttop2vec, which leverages joint document and word semantic embedding to find textittopic vectors. This model does not require stop-word lists, stemming or lemmatization, and it automatically finds the number of topics. The resulting topic vectors are jointly embedded with the document and word vectors with distance between them representing semantic similarity. Our experiments demonstrate that texttttop2vec finds topics which are significantly more informative and representative of the corpus trained on than probabilistic generative models.