mlsanityData to 7 Oct 2026

Improving Neural Topic Models using Knowledge Distillation

arXiv 2010.02377 ↗First posted 5 Oct 2020cs.CL cs.IR cs.LGcited by 25

Not on the map. Why?

Abstract (arXiv)

Topic models are often used to identify human-interpretable topics to help make sense of large document collections. We use knowledge distillation to combine the best attributes of probabilistic topic models and pretrained transformers. Our modular method can be straightforwardly applied with any neural topic model to improve topic quality, which we demonstrate using two models having disparate architectures, obtaining state-of-the-art topic coherence. We show that our adaptable framework not only improves performance in the aggregate over all estimated topics, as is commonly reported, but also in head-to-head comparisons of aligned topics.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 4; 2023: 7; 2024: 3; 2025: 2; 2026: 2.

25 papers.

Sort:

  1. 2021Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherencecited by 74
  2. 2021Topic Modelling Meets Deep Neural Networks: A Surveycited by 51
  3. 2021Contrastive Learning for Neural Topic Modelcited by 48
  4. 2022Topic Discovery via Latent Space Clustering of Pretrained Language Model Representationscited by 34
  5. 2023Prompting Large Language Models for Topic Modelingcited by 24
  6. 2024A Survey on Neural Topic Models: Methods, Applications, and Challengescited by 18
  7. 2024Topic Modeling as Multi-Objective Contrastive Optimizationcited by 12
  8. 2022Are Neural Topic Models Broken?cited by 11
  9. 2021Deep Transfer Learning & Beyond: Transformer Language Models in Information Systems Researchcited by 7
  10. 2023CWTM: Leveraging Contextualized Word Embeddings from BERT for Neural Topic Modelingcited by 7
  11. 2021Neural Attention-Aware Hierarchical Topic Modelcited by 6
  12. 2022Multilingual and Multimodal Topic Modelling with Pretrained Embeddingscited by 6
13 more
  1. 2022Knowledge-Aware Bayesian Deep Topic Modelcited by 5
  2. 2023Diversity-Aware Coherence Loss for Improving Neural Topic Modelscited by 5
  3. 2023KDSTM: Neural Semi-supervised Topic Modeling with Knowledge Distillationcited by 3
  4. 2025XTRA: Cross-Lingual Topic Modeling with Topic and Representation Alignmentscited by 3
  5. 2024Topic Modeling with Fine-tuning LLMs and Bag of Sentencescited by 2
  6. 2021Beyond Topics: Discovering Latent Healthcare Objectives from Event Sequencescited by 1
  7. 2021Survey of Generative Methods for Social Media Analysiscited by 1
  8. 2023Efficient and Flexible Topic Modeling using Pretrained Embeddings and Bag of Sentencescited by 1
  9. 2023Graph Contrastive Topic Modelcited by 1
  10. 2023FATRER: Full-Attention Topic Regularizer for Accurate and Robust Conversational Emotion Recognitioncited by 1
  11. 2026PRISM: LLM-Guided Semantic Clustering for High-Precision Topicscited by 1
  12. 2026LLM-XTM: Enhancing Cross-Lingual Topic Models with Large Language Modelscited by 1
  13. 2025Self-Supervised Learning for Neural Topic Models with Variance-Invariance-Covariance Regularizationcited by 0