mlsanityData to 7 Oct 2026

Is Neural Topic Modelling Better than Clustering? An Empirical Study on Clustering with Contextual Embeddings for Topics

arXiv 2204.09874 ↗First posted 21 Apr 2022cs.CLcited by 35

Not on the map. Why?

Abstract (arXiv)

Recent work incorporates pre-trained word embeddings such as BERT embeddings into Neural Topic Models (NTMs), generating highly coherent topics. However, with high-quality contextualized document representations, do we really need sophisticated neural models to obtain coherent and interpretable topics? In this paper, we conduct thorough experiments showing that directly clustering high-quality sentence embeddings with an appropriate word selecting method can generate more coherent and diverse topics than NTMs, achieving also higher efficiency and simplicity.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Builds on 18 of its 28 references that are in the map. First posted on arXiv 21 Apr 2022.

Builds on

18 of 28 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  5. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  6. 2017Autoencoding Variational Inference For Topic Modelscited by 168
  7. 2020Unsupervised Domain Clusters in Pretrained Language Modelscited by 160
  8. 2020Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherencecited by 121
  9. 2015Neural Variational Inference for Text Processingcited by 111
  10. 2017Accelerated Hierarchical Density Clusteringcited by 100
  11. 2020Tired of Topic Models? Clusters of Pretrained Word Embeddings Make for Fast and Good Topics too!cited by 73
  12. 2020Cross-lingual Contextualized Topic Models with Zero-shot Learningcited by 66
6 more
  1. 2019Topic Modeling with Wasserstein Autoencoderscited by 53
  2. 2021Topic Modelling Meets Deep Neural Networks: A Surveycited by 51
  3. 2020Topic Modeling with Contextualized Word Representation Clusterscited by 41
  4. 2020Which *BERT? A Survey Organizing Contextualized Encoderscited by 26
  5. 2021Neural Attention-Aware Hierarchical Topic Modelcited by 6
  6. 2016Bibliographic Analysis with the Citation Network Topic Modelcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 10; 2024: 11; 2025: 10; 2026: 2.

35 papers.

Sort:

By area:

  1. 2023How Do Transformers Learn Topic Structure: Towards a Mechanistic Understandingcited by 85
  2. 2023Effective Neural Topic Modeling with Embedding Clustering Regularizationcited by 46
  3. 2024FASTopic: Pretrained Transformer is a Fast, Adaptive, Stable, and Transferable Topic Modelcited by 34
  4. 2024On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMscited by 24
  5. 2023Goodtriever: Adaptive Toxicity Mitigation with Retrieval-augmented Modelscited by 23
  6. 2024A Survey on Neural Topic Models: Methods, Applications, and Challengescited by 18
  7. 2023DeTiME: Diffusion-Enhanced Topic Modeling using Encoder-decoder based LLMcited by 16
  8. 2022Effective Seed-Guided Topic Discovery by Integrating Multiple Types of Contextscited by 12
  9. 2024NeuroMax: Enhancing Neural Topic Modeling via Maximizing Mutual Information and Group Topic Regularizationcited by 9
  10. 2024Semantic-Driven Topic Modeling Using Transformer-Based Embeddings and Clustering Algorithmscited by 8
  11. 2025A Survey on Retrieval And Structuring Augmented Generation with Large Language Modelscited by 8
  12. 2023Deep de Finetti: Recovering Topic Distributions from Large Language Modelscited by 7
23 more
  1. 2024Morality is Non-Binary: Building a Pluralist Moral Sentence Embedding Space using Contrastive Learningcited by 7
  2. 2025Model Directions, Not Words: Mechanistic Topic Models Using Sparse Autoencoderscited by 7
  3. 2024Recent Trends in Linear Text Segmentation: a Surveycited by 6
  4. 2023Topics as Entity Clusters: Entity-based Topics from Large Language Models and Graph Neural Networkscited by 3
  5. 2023Lady and the Tramp Nextdoor: Online Manifestations of Economic Inequalities in the Nextdoor Social Networkcited by 3
  6. 2023REAL: A Representative Error-Driven Approach for Active Learningcited by 3
  7. 2024Comprehensive Evaluation of Large Language Models for Topic Modelingcited by 3
  8. 2024Refining Dimensions for Improving Clustering-based Cross-lingual Topic Modelscited by 3
  9. 2025ProxAnn: Use-Oriented Evaluations of Topic Models and Document Clusteringcited by 3
  10. 2025Sparse Autoencoders are Topic Modelscited by 3
  11. 2024TroubleLLM: Align to Red Team Expertcited by 2
  12. 2025Large Language Models for History, Philosophy, and Sociology of Science: Interpretive Uses, Methodological Challenges, and Critical Perspectivescited by 2
  13. 2024Variational Language Concepts for Interpreting Foundation Language Modelscited by 1
  14. 2025The Medium Is Not the Message: Deconfounding Document Embeddings via Linear Concept Erasurecited by 1
  15. 2022Moving beyond word lists: towards abstractive topic labels for human-like topics of scientific documentscited by 0
  16. 2023Graph2topic: an opensource topic modeling framework based on sentence embedding and community detectioncited by 0
  17. 2023Constructing and Interpreting Causal Knowledge Graphs from Newscited by 0
  18. 2025Multilingualism, Transnationality, and K-pop in the Online #StopAsianHate Movementcited by 0
  19. 2025Document clustering with evolved multiword search queriescited by 0
  20. 2025The Efficiency of Pre-training with Objective Masking in Pseudo Labeling for Semi-Supervised Text Classificationcited by 0
  21. 2025CATCH: A Controllable Theme Detection Framework with Contextualized Clustering and Hierarchical Generationcited by 0
  22. 2026Improving Topic Modeling by Distilling Soft Labels from Language Modelscited by 0
  23. 2026Comment-level Topic Drift Analysis in the Reddit Corpuscited by 0