# Comparison of Topic Modelling Approaches in the Banking Context

arXiv: https://arxiv.org/abs/2402.03176 · HTML: https://mlsanity.com/paper/2402.03176/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 65 references that are in the map.
- First posted on arXiv 5 Feb 2024.

## Builds on (most cited first, 10 of 10)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [FinBERT: Financial Sentiment Analysis with Pre-trained Language Models](https://mlsanity.com/paper/1908.10063/) (2019)
- [Topic-Driven and Knowledge-Aware Transformer for Dialogue Emotion Detection](https://mlsanity.com/paper/2106.01071/) (2021)
- [Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations](https://mlsanity.com/paper/2202.04582/) (2022)
- [What is Wrong with Topic Modeling? (and How to Fix it Using Search-based Software Engineering)](https://mlsanity.com/paper/1608.08176/) (2016)
- [Latent Dirichlet Allocation (LDA) for Topic Modeling of the CFPB Consumer Complaints](https://mlsanity.com/paper/1807.07468/) (2018)
- [Topic Modelling on Consumer Financial Protection Bureau Data: An Approach Using BERT Based Embeddings](https://mlsanity.com/paper/2205.07259/) (2022)

## Built on by (most cited first, 10 of 10)

- [When Your Reviewer is an LLM: Biases, Divergence, and Prompt Injection Risks in Peer Review](https://mlsanity.com/paper/2509.09912/) (2025)
- [The Use of a Large Language Model for Cyberbullying Detection](https://mlsanity.com/paper/2402.04088/) (2024)
- [An Exploration of Clustering Algorithms for Customer Segmentation in the UK Retail Market](https://mlsanity.com/paper/2402.04103/) (2024)
- [A Systematic Review of Generative AI for Teaching and Learning Practice](https://mlsanity.com/paper/2406.09520/) (2024)
- [Recommending the right academic programs: An interest mining approach using BERTopic](https://mlsanity.com/paper/2501.06581/) (2025)
- [Analyzing German Parliamentary Speeches: A Machine Learning Approach for Topic and Sentiment Classification](https://mlsanity.com/paper/2508.03181/) (2025)
- [LLM-Assisted Topic Reduction for BERTopic on Social Media Data](https://mlsanity.com/paper/2509.19365/) (2025)
- [An Iterative Approach to Topic Modelling](https://mlsanity.com/paper/2407.17892/) (2024)
- [Enhancing BERTopic with Intermediate Layer Representations](https://mlsanity.com/paper/2505.06696/) (2025)
- [Distributional sentiment modeling and anomaly detection for consumer complaint assessment](https://mlsanity.com/paper/2609.31653/) (2026)

## Abstract (arXiv)

Topic modelling is a prominent task for automatic topic extraction in many applications such as sentiment analysis and recommendation systems. The approach is vital for service industries to monitor their customer discussions. The use of traditional approaches such as Latent Dirichlet Allocation (LDA) for topic discovery has shown great performances, however, they are not consistent in their results as these approaches suffer from data sparseness and inability to model the word order in a document. Thus, this study presents the use of Kernel Principal Component Analysis (KernelPCA) and K-means Clustering in the BERTopic architecture. We have prepared a new dataset using tweets from customers of Nigerian banks and we use this to compare the topic modelling approaches. Our findings showed KernelPCA and K-means in the BERTopic architecture-produced coherent topics with a coherence score of 0.8463.