mlsanityData to 7 Oct 2026

Contextualized Topic Coherence Metrics

arXiv 2305.14587 ↗First posted 23 May 2023cs.CL cs.IRcited by 19

Not on the map. Why?

Abstract (arXiv)

The recent explosion in work on neural topic modeling has been criticized for optimizing automated topic evaluation metrics at the expense of actual meaningful topic identification. But human annotation remains expensive and time-consuming. We propose LLM-based methods inspired by standard human topic evaluations, in a family of metrics called Contextualized Topic Coherence (CTC). We evaluate both a fully automated version as well as a semi-automated CTC that allows human-centered evaluation of coherence while maintaining the efficiency of automated methods. We evaluate CTC relative to five other metrics on six topic models and find that it outperforms automated topic coherence methods, works well on short documents, and is not susceptible to meaningless but high-scoring topics.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 16 of its 52 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

16 of 52 references.

References · 16

  1. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  2. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  3. 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 779
  4. 2014Distributed Representations of Sentences and Documentscited by 677
  5. 2019Topic Modeling in Embedding Spacescited by 224
  6. 2017Autoencoding Variational Inference For Topic Modelscited by 168
  7. 2020Top2Vec: Distributed Representations of Topicscited by 124
  8. 2020Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherencecited by 121
  9. 2021Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherencecited by 74
  10. 2021Topic Modelling Meets Deep Neural Networks: A Surveycited by 51
  11. 2020Topic Modeling with Contextualized Word Representation Clusterscited by 41
  12. 2018ATM:Adversarial-neural Topic Modelcited by 23
4 more
  1. 2022Are Neural Topic Models Broken?cited by 11
  2. 2021Linguistic Dependencies and Statistical Dependencecited by 10
  3. 2023ANTM: An Aligned Neural Topic Model for Exploring Evolving Topicscited by 6
  4. 2019Automatic Evaluation of Local Topic Qualitycited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 6; 2026: 5.

19 papers.

Sort:

By area:

  1. 2024LLM Reading Tea Leaves: Automatically Evaluating Topic Models with Large Language Modelscited by 11
  2. 2024Neural Topic Modeling with Large Language Models in the Loopcited by 10
  3. 2024A Computational Method for Measuring "Open Codes" in Qualitative Analysiscited by 6
  4. 2024PROMPTHEUS: A Human-Centered Pipeline to Streamline SLRs with LLMscited by 4
  5. 2025Bridging the Evaluation Gap: Leveraging Large Language Models for Topic Model Evaluationcited by 4
  6. 2024CAST: Corpus-Aware Self-similarity Enhanced Topic modellingcited by 3
  7. 2025ProxAnn: Use-Oriented Evaluations of Topic Models and Document Clusteringcited by 3
  8. 2025Sparse Autoencoders are Topic Modelscited by 3
  9. 2024Unveiling the Potential of BERTopic for Multilingual Fake News Analysis -- Use Case: Covid-19cited by 1
  10. 2024Multi-Faceted Question Complexity Estimation Targeting Topic Domain-Specificitycited by 1
  11. 2025Toward Purpose-oriented Topic Model Evaluation enabled by Large Language Modelscited by 1
  12. 2024S3 -- Semantic Signal Separationcited by 0
7 more
  1. 2025Contextual Embedding-based Clustering to Identify Topics for Healthcare Service Improvementcited by 0
  2. 2025Objectifying the Subjective: Cognitive Biases in Topic Interpretationscited by 0
  3. 2026Improving Topic Modeling by Distilling Soft Labels from Language Modelscited by 0
  4. 2026Disentangling Similarity and Relatedness in Topic Modelscited by 0
  5. 2026From Documents to Segments: A Contextual Reformulation for Topic Assignmentcited by 0
  6. 2026How Do AI Coding Agents Contribute to Software Development? an Empirical Study of Agentic Pull Requestscited by 0
  7. 2026BERTilda: Explainable Topic Lifecycle Tracking with Split/Merge Detection via Similarity-and-Flow Temporal Graphscited by 0