# Topic Modeling over Short Texts by Incorporating Word Embeddings

arXiv: https://arxiv.org/abs/1609.08496 · HTML: https://mlsanity.com/paper/1609.08496/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Sep 2016.

## Built on by (most cited first, 9 of 9)

- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Experiments on Generalizability of BERTopic on Multi-Domain Short Text](https://mlsanity.com/paper/2212.08459/) (2022)
- [Semantic-Driven Topic Modeling Using Transformer-Based Embeddings and Clustering Algorithms](https://mlsanity.com/paper/2410.00134/) (2024)
- [Artificial intelligence for Sustainable Energy: A Contextual Topic Modeling and Content Analysis](https://mlsanity.com/paper/2110.00828/) (2021)
- [Semantic-Driven Topic Modeling for Analyzing Creativity in Virtual Brainstorming](https://mlsanity.com/paper/2509.16835/) (2025)
- [An Embedding-based Joint Sentiment-Topic Model for Short Texts](https://mlsanity.com/paper/2103.14410/) (2021)
- [Semantic Component Analysis: Introducing Multi-Topic Distributions to Clustering-Based Topic Modeling](https://mlsanity.com/paper/2410.21054/) (2024)
- [TopiCLEAR: Adaptive embedding clustering for interpretable topic discovery from short texts](https://mlsanity.com/paper/2512.06694/) (2025)
- [MMTM: Tri-Modal Topic Modeling for Long-Form Video via Similarity-Gated Fusion](https://mlsanity.com/paper/2605.29765/) (2026)

## Abstract (arXiv)

Inferring topics from the overwhelming amount of short texts becomes a critical but challenging task for many content analysis tasks, such as content charactering, user interest profiling, and emerging topic detecting. Existing methods such as probabilistic latent semantic analysis (PLSA) and latent Dirichlet allocation (LDA) cannot solve this prob- lem very well since only very limited word co-occurrence information is available in short texts. This paper studies how to incorporate the external word correlation knowledge into short texts to improve the coherence of topic modeling. Based on recent results in word embeddings that learn se- mantically representations for words from a large corpus, we introduce a novel method, Embedding-based Topic Model (ETM), to learn latent topics from short texts. ETM not only solves the problem of very limited word co-occurrence information by aggregating short texts into long pseudo- texts, but also utilizes a Markov Random Field regularized model that gives correlated words a better chance to be put into the same topic. The experiments on real-world datasets validate the effectiveness of our model comparing with the state-of-the-art models.