# Human-interpretable clustering of short-text using large language models

arXiv: https://arxiv.org/abs/2405.07278 · HTML: https://mlsanity.com/paper/2405.07278/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 69 references that are in the map.
- First posted on arXiv 12 May 2024.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models](https://mlsanity.com/paper/2304.01852/) (2023)
- [Latent Dirichlet Allocation (LDA) and Topic modeling: models, applications, a survey](https://mlsanity.com/paper/1711.04305/) (2017)
- [Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherence](https://mlsanity.com/paper/2107.02173/) (2021)
- [Tired of Topic Models? Clusters of Pretrained Word Embeddings Make for Fast and Good Topics too!](https://mlsanity.com/paper/2004.14914/) (2020)
- [Enhancing Clinical Concept Extraction with Contextual Embeddings](https://mlsanity.com/paper/1902.08691/) (2019)
- [Short Text Topic Modeling Techniques, Applications, and Performance: A Survey](https://mlsanity.com/paper/1904.07695/) (2019)
- [Clustering Algorithms: A Comparative Approach](https://mlsanity.com/paper/1612.08388/) (2016)
- [Applying BERT and ChatGPT for Sentiment Analysis of Lyme Disease in Scientific Literature](https://mlsanity.com/paper/2302.06474/) (2023)
- [Comparative Analysis of Word Embeddings for Capturing Word Similarities](https://mlsanity.com/paper/2005.03812/) (2020)
- [Searching for superspreaders of information in real-world social media](https://mlsanity.com/paper/1405.1790/) (2014)
- [Quantifying the Dissimilarity of Texts](https://mlsanity.com/paper/2305.02457/) (2023)

## Built on by (most cited first, 3 of 3)

- [HERCULES: Hierarchical Embedding-based Recursive Clustering Using LLMs for Efficient Summarization](https://mlsanity.com/paper/2506.19992/) (2025)
- [Moving Past Single Metrics: Exploring Short-Text Clustering Across Multiple Resolutions](https://mlsanity.com/paper/2502.17020/) (2025)
- [Balancing Complexity and Informativeness in LLM-Based Clustering: Finding the Goldilocks Zone](https://mlsanity.com/paper/2504.04314/) (2025)

## Abstract (arXiv)

Clustering short text is a difficult problem, due to the low word co-occurrence between short text documents. This work shows that large language models (LLMs) can overcome the limitations of traditional clustering approaches by generating embeddings that capture the semantic nuances of short text. In this study clusters are found in the embedding space using Gaussian Mixture Modelling (GMM). The resulting clusters are found to be more distinctive and more human-interpretable than clusters produced using the popular methods of doc2vec and Latent Dirichlet Allocation (LDA). The success of the clustering approach is quantified using human reviewers and through the use of a generative LLM. The generative LLM shows good agreement with the human reviewers, and is suggested as a means to bridge the `validation gap' which often exists between cluster production and cluster interpretation. The comparison between LLM-coding and human-coding reveals intrinsic biases in each, challenging the conventional reliance on human coding as the definitive standard for cluster validation.