# Semantic-Driven Topic Modeling Using Transformer-Based Embeddings and Clustering Algorithms

arXiv: https://arxiv.org/abs/2410.00134 · HTML: https://mlsanity.com/paper/2410.00134/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 39 references that are in the map.
- First posted on arXiv 30 Sep 2024.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Topic Modeling in Embedding Spaces](https://mlsanity.com/paper/1907.04907/) (2019)
- [Top2Vec: Distributed Representations of Topics](https://mlsanity.com/paper/2008.09470/) (2020)
- [Cross-lingual Contextualized Topic Models with Zero-shot Learning](https://mlsanity.com/paper/2004.07737/) (2020)
- [Topic Modelling Meets Deep Neural Networks: A Survey](https://mlsanity.com/paper/2103.00498/) (2021)
- [Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Direction](https://mlsanity.com/paper/2409.00265/) (2024)
- [Is Neural Topic Modelling Better than Clustering? An Empirical Study on Clustering with Contextual Embeddings for Topics](https://mlsanity.com/paper/2204.09874/) (2022)
- [Topic Modeling over Short Texts by Incorporating Word Embeddings](https://mlsanity.com/paper/1609.08496/) (2016)

## Built on by (most cited first, 8 of 8)

- [Ethio-Fake: Cutting-Edge Approaches to Combat Fake News in Under-Resourced Languages Using Explainable AI](https://mlsanity.com/paper/2410.02609/) (2024)
- [A Unified Framework with Novel Metrics for Evaluating the Effectiveness of XAI Techniques in LLMs](https://mlsanity.com/paper/2503.05050/) (2025)
- [Explainable AI: XAI-Guided Context-Aware Data Augmentation](https://mlsanity.com/paper/2506.03484/) (2025)
- [Toxicity in Online Platforms and AI Systems: A Survey of Needs, Challenges, Mitigations, and Future Directions](https://mlsanity.com/paper/2509.25539/) (2025)
- [Semantic-Driven Topic Modeling for Analyzing Creativity in Virtual Brainstorming](https://mlsanity.com/paper/2509.16835/) (2025)
- [Bilingual Word Level Language Identification for Omotic Languages](https://mlsanity.com/paper/2509.07998/) (2025)
- [Automated Unity Game Template Generation from GDDs via NLP and Multi-Modal LLMs](https://mlsanity.com/paper/2509.08847/) (2025)
- [Explainable AI: Context-Aware Layer-Wise Integrated Gradients for Explaining Transformer Models](https://mlsanity.com/paper/2602.16608/) (2026)

## Abstract (arXiv)

Topic modeling is a powerful technique to discover hidden topics and patterns within a collection of documents without prior knowledge. Traditional topic modeling and clustering-based techniques encounter challenges in capturing contextual semantic information. This study introduces an innovative end-to-end semantic-driven topic modeling technique for the topic extraction process, utilizing advanced word and document embeddings combined with a powerful clustering algorithm. This semantic-driven approach represents a significant advancement in topic modeling methodologies. It leverages contextual semantic information to extract coherent and meaningful topics. Specifically, our model generates document embeddings using pre-trained transformer-based language models, reduces the dimensions of the embeddings, clusters the embeddings based on semantic similarity, and generates coherent topics for each cluster. Compared to ChatGPT and traditional topic modeling algorithms, our model provides more coherent and meaningful topics.