# Multilingual transformer and BERTopic for short text topic modeling: The case of Serbian

arXiv: https://arxiv.org/abs/2402.03067 · HTML: https://mlsanity.com/paper/2402.03067/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 22 references that are in the map.
- First posted on arXiv 5 Feb 2024.

## Builds on (most cited first, 6 of 6)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Topic Modeling in Embedding Spaces](https://mlsanity.com/paper/1907.04907/) (2019)
- [BERTi'c -- The Transformer Language Model for Bosnian, Croatian, Montenegrin and Serbian](https://mlsanity.com/paper/2104.09243/) (2021)
- [Experiments on Generalizability of BERTopic on Multi-Domain Short Text](https://mlsanity.com/paper/2212.08459/) (2022)
- [Topic Modelling on Consumer Financial Protection Bureau Data: An Approach Using BERT Based Embeddings](https://mlsanity.com/paper/2205.07259/) (2022)

## Built on by (most cited first, 6 of 6)

- [BERTopic for Topic Modeling of Hindi Short Texts: A Comparative Study](https://mlsanity.com/paper/2501.03843/) (2025)
- [What Challenges Do Developers Face in AI Agent Systems? An Empirical Study on Stack Overflow & GitHub Issues](https://mlsanity.com/paper/2510.25423/) (2025)
- [Improving Customer Service with Automatic Topic Detection in User Emails](https://mlsanity.com/paper/2502.19115/) (2025)
- [WildFireCan-MMD: A Multimodal Dataset for Classification of User-Generated Content During Wildfires in Canada](https://mlsanity.com/paper/2504.13231/) (2025)
- [Holistic Evaluations of Topic Models](https://mlsanity.com/paper/2507.23364/) (2025)
- [Automating Historical Insight Extraction from Large-Scale Newspaper Archives via Neural Topic Modeling](https://mlsanity.com/paper/2512.11635/) (2025)

## Abstract (arXiv)

This paper presents the results of the first application of BERTopic, a state-of-the-art topic modeling technique, to short text written in a morphologi-cally rich language. We applied BERTopic with three multilingual embed-ding models on two levels of text preprocessing (partial and full) to evalu-ate its performance on partially preprocessed short text in Serbian. We also compared it to LDA and NMF on fully preprocessed text. The experiments were conducted on a dataset of tweets expressing hesitancy toward COVID-19 vaccination. Our results show that with adequate parameter setting, BERTopic can yield informative topics even when applied to partially pre-processed short text. When the same parameters are applied in both prepro-cessing scenarios, the performance drop on partially preprocessed text is minimal. Compared to LDA and NMF, judging by the keywords, BERTopic offers more informative topics and gives novel insights when the number of topics is not limited. The findings of this paper can be significant for re-searchers working with other morphologically rich low-resource languages and short text.