# LLM-Assisted Topic Reduction for BERTopic on Social Media Data

arXiv: https://arxiv.org/abs/2509.19365 · HTML: https://mlsanity.com/paper/2509.19365/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 25 references that are in the map.
- First posted on arXiv 18 Sep 2025.

## Builds on (most cited first, 9 of 9)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Short Text Topic Modeling Techniques, Applications, and Performance: A Survey](https://mlsanity.com/paper/1904.07695/) (2019)
- [Prompting Large Language Models for Topic Modeling](https://mlsanity.com/paper/2312.09693/) (2023)
- [Comparison of Topic Modelling Approaches in the Banking Context](https://mlsanity.com/paper/2402.03176/) (2024)
- [Experiments on Generalizability of BERTopic on Multi-Domain Short Text](https://mlsanity.com/paper/2212.08459/) (2022)
- [AHAM: Adapt, Help, Ask, Model -- Harvesting LLMs for literature mining](https://mlsanity.com/paper/2312.15784/) (2023)
- [Revisiting Automated Topic Model Evaluation with Large Language Models](https://mlsanity.com/paper/2305.12152/) (2023)

## Built on by (most cited first, 1 of 1)

- [LLM Serving in the Wild: An Empirical Study of Frameworks, Methods, and System Designs](https://mlsanity.com/paper/2608.03036/) (2026)

## Abstract (arXiv)

The BERTopic framework leverages transformer embeddings and hierarchical clustering to extract latent topics from unstructured text corpora. While effective, it often struggles with social media data, which tends to be noisy and sparse, resulting in an excessive number of overlapping topics. Recent work explored the use of large language models for end-to-end topic modelling. However, these approaches typically require significant computational overhead, limiting their scalability in big data contexts. In this work, we propose a framework that combines BERTopic for topic generation with large language models for topic reduction. The method first generates an initial set of topics and constructs a representation for each. These representations are then provided as input to the language model, which iteratively identifies and merges semantically similar topics. We evaluate the approach across three Twitter/X datasets and four different language models. Our method outperforms the baseline approach in enhancing topic diversity and, in many cases, coherence, with some sensitivity to dataset characteristics and initial parameter selection.