# Contextual Embedding-based Clustering to Identify Topics for Healthcare Service Improvement

arXiv: https://arxiv.org/abs/2504.14068 · HTML: https://mlsanity.com/paper/2504.14068/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 45 references that are in the map.
- First posted on arXiv 18 Apr 2025.

## Builds on (most cited first, 15 of 15)

- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Topic Modeling in Embedding Spaces](https://mlsanity.com/paper/1907.04907/) (2019)
- [Tired of Topic Models? Clusters of Pretrained Word Embeddings Make for Fast and Good Topics too!](https://mlsanity.com/paper/2004.14914/) (2020)
- [Unveiling Security, Privacy, and Ethical Concerns of ChatGPT](https://mlsanity.com/paper/2307.14192/) (2023)
- [A Survey on Contextual Embeddings](https://mlsanity.com/paper/2003.07278/) (2020)
- [Topic Modeling with Contextualized Word Representation Clusters](https://mlsanity.com/paper/2010.12626/) (2020)
- [Stop Clickbait: Detecting and Preventing Clickbaits in Online News Media](https://mlsanity.com/paper/1610.09786/) (2016)
- [Simple and Effective Dimensionality Reduction for Word Embeddings](https://mlsanity.com/paper/1708.03629/) (2017)
- [Large Language Models Offer an Alternative to the Traditional Approach of Topic Modelling](https://mlsanity.com/paper/2403.16248/) (2024)
- [Automatically Identifying Complaints in Social Media](https://mlsanity.com/paper/1906.03890/) (2019)
- [Contextualized Topic Coherence Metrics](https://mlsanity.com/paper/2305.14587/) (2023)
- [GPTopic: Dynamic and Interactive Topic Representations](https://mlsanity.com/paper/2403.03628/) (2024)
- [Addressing Topic Granularity and Hallucination in Large Language Models for Topic Modelling](https://mlsanity.com/paper/2405.00611/) (2024)
- [How Reliable AI Chatbots are for Disease Prediction from Patient Complaints?](https://mlsanity.com/paper/2405.13219/) (2024)
- [Computational Analysis of Insurance Complaints: GEICO Case Study](https://mlsanity.com/paper/1806.09736/) (2018)

## Abstract (arXiv)

Understanding patient feedback is crucial for improving healthcare services, yet analyzing unlabeled short-text feedback presents challenges due to limited data and domain-specific nuances. Traditional supervised approaches require extensive labeled datasets, making unsupervised methods more practical for extracting insights. This study applies unsupervised techniques to analyze 439 survey responses from a healthcare system in Wisconsin, USA. A keyword-based filter was used to isolate complaint-related feedback using a domain-specific lexicon. To identify dominant themes, we evaluated traditional topic models such as Latent Dirichlet Allocation (LDA) and Gibbs Sampling Dirichlet Multinomial Mixture (GSDMM) -- alongside BERTopic, a neural embedding-based clustering method. To improve coherence and interpretability in sparse, short-text data, we propose kBERT, which integrates BERT embeddings with k-means clustering. Model performance was assessed using coherence scores (Cv ) and average Inverted Rank-Biased Overlap (IRBOavg). kBERT achieved the highest coherence (Cv = 0.53) and topic separation (IRBOavg = 1.00), outperforming all other models. These findings highlight the value of embedding-based, context-aware models in healthcare analytics.