# A Topic Coverage Approach to Evaluation of Topic Models

arXiv: https://arxiv.org/abs/2012.06274 · HTML: https://mlsanity.com/paper/2012.06274/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Dec 2020.

## Built on by (most cited first, 7 of 7)

- [Are Neural Topic Models Broken?](https://mlsanity.com/paper/2210.16162/) (2022)
- [Bridging the Evaluation Gap: Leveraging Large Language Models for Topic Model Evaluation](https://mlsanity.com/paper/2502.07352/) (2025)
- [Toward Purpose-oriented Topic Model Evaluation enabled by Large Language Models](https://mlsanity.com/paper/2509.07142/) (2025)
- [TopicGPT: A Prompt-based Topic Modeling Framework](https://mlsanity.com/paper/2311.01449/) (2023)
- [Investigating Thematic Patterns and User Preferences in LLM Interactions using BERTopic](https://mlsanity.com/paper/2510.07557/) (2025)
- [Iterative Improvement of an Additively Regularized Topic Model](https://mlsanity.com/paper/2408.05840/) (2024)
- [Label Semantic Expansion via Label Guided Neural Topic Modeling](https://mlsanity.com/paper/2608.30216/) (2026)

## Abstract (arXiv)

Topic models are widely used unsupervised models capable of learning topics - weighted lists of words and documents - from large collections of text documents. When topic models are used for discovery of topics in text collections, a question that arises naturally is how well the model-induced topics correspond to topics of interest to the analyst. In this paper we revisit and extend a so far neglected approach to topic model evaluation based on measuring topic coverage - computationally matching model topics with a set of reference topics that models are expected to uncover. The approach is well suited for analyzing models' performance in topic discovery and for large-scale analysis of both topic models and measures of model quality. We propose new measures of coverage and evaluate, in a series of experiments, different types of topic models on two distinct text domains for which interest for topic discovery exists. The experiments include evaluation of model quality, analysis of coverage of distinct topic categories, and the analysis of the relationship between coverage and other methods of topic model evaluation. The paper contributes a new supervised measure of coverage, and the first unsupervised measure of coverage. The supervised measure achieves topic matching accuracy close to human agreement. The unsupervised measure correlates highly with the supervised one (Spearman's ρ ≥ 0.95). Other contributions include insights into both topic models and different methods of model evaluation, and the datasets and code for facilitating future research on topic coverage.