# Bridging the Evaluation Gap: Leveraging Large Language Models for Topic Model Evaluation

arXiv: https://arxiv.org/abs/2502.07352 · HTML: https://mlsanity.com/paper/2502.07352/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 57 references that are in the map.
- First posted on arXiv 11 Feb 2025.

## Builds on (most cited first, 24 of 24)

- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Topic Modeling in Embedding Spaces](https://mlsanity.com/paper/1907.04907/) (2019)
- [Autoencoding Variational Inference For Topic Models](https://mlsanity.com/paper/1703.01488/) (2017)
- [Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherence](https://mlsanity.com/paper/2004.03974/) (2020)
- [Neural Variational Inference for Text Processing](https://mlsanity.com/paper/1511.06038/) (2015)
- [Discovering Discrete Latent Topics with Neural Variational Inference](https://mlsanity.com/paper/1706.00359/) (2017)
- [Tired of Topic Models? Clusters of Pretrained Word Embeddings Make for Fast and Good Topics too!](https://mlsanity.com/paper/2004.14914/) (2020)
- [Cross-lingual Contextualized Topic Models with Zero-shot Learning](https://mlsanity.com/paper/2004.07737/) (2020)
- [Topic Modeling with Wasserstein Autoencoders](https://mlsanity.com/paper/1907.12374/) (2019)
- [Neural Models for Documents with Metadata](https://mlsanity.com/paper/1705.09296/) (2017)
- [Effective Neural Topic Modeling with Embedding Clustering Regularization](https://mlsanity.com/paper/2306.04217/) (2023)
- [Segment Any Text: A Universal Approach for Robust, Efficient and Adaptable Sentence Segmentation](https://mlsanity.com/paper/2406.16678/) (2024)
- [Large Language Models Offer an Alternative to the Traditional Approach of Topic Modelling](https://mlsanity.com/paper/2403.16248/) (2024)
- [WHAI: Weibull Hybrid Autoencoding Inference for Deep Topic Modeling](https://mlsanity.com/paper/1803.01328/) (2018)
- [Neural Topic Modeling with Bidirectional Adversarial Training](https://mlsanity.com/paper/2004.12331/) (2020)
- [Coherence-Aware Neural Topic Modeling](https://mlsanity.com/paper/1809.02687/) (2018)
- [ATM:Adversarial-neural Topic Model](https://mlsanity.com/paper/1811.00265/) (2018)
- [Transformer models: an introduction and catalog](https://mlsanity.com/paper/2302.07730/) (2023)
- [Contextualized Topic Coherence Metrics](https://mlsanity.com/paper/2305.14587/) (2023)
- [An Automatic Approach for Document-level Topic Model Evaluation](https://mlsanity.com/paper/1706.05140/) (2017)
- [LLM Reading Tea Leaves: Automatically Evaluating Topic Models with Large Language Models](https://mlsanity.com/paper/2406.09008/) (2024)
- [A Topic Coverage Approach to Evaluation of Topic Models](https://mlsanity.com/paper/2012.06274/) (2020)
- [Addressing Topic Granularity and Hallucination in Large Language Models for Topic Modelling](https://mlsanity.com/paper/2405.00611/) (2024)
- [RaTE: a Reproducible automatic Taxonomy Evaluation by Filling the Gap](https://mlsanity.com/paper/2307.09706/) (2023)

## Built on by (most cited first, 4 of 4)

- [Toward Purpose-oriented Topic Model Evaluation enabled by Large Language Models](https://mlsanity.com/paper/2509.07142/) (2025)
- [LLM as Attention-Informed NTM and Topic Modeling as long-input Generation: Interpretability and long-Context Capability](https://mlsanity.com/paper/2510.03174/) (2025)
- [TriTopic: Tri-Modal Graph-Based Topic Modeling with Iterative Refinement and Archetypes](https://mlsanity.com/paper/2602.19079/) (2026)
- [EviMap: Evidence-Grounded Hierarchical Topic Maps for Exploring Unlabeled Corpora](https://mlsanity.com/paper/2609.06664/) (2026)

## Abstract (arXiv)

This study presents a framework for automated evaluation of dynamically evolving topic taxonomies in scientific literature using Large Language Models (LLMs). In digital library systems, topic modeling plays a crucial role in efficiently organizing and retrieving scholarly content, guiding researchers through complex knowledge landscapes. As research domains proliferate and shift, traditional human centric and static evaluation methods struggle to maintain relevance. The proposed approach harnesses LLMs to measure key quality dimensions, such as coherence, repetitiveness, diversity, and topic-document alignment, without heavy reliance on expert annotators or narrow statistical metrics. Tailored prompts guide LLM assessments, ensuring consistent and interpretable evaluations across various datasets and modeling techniques. Experiments on benchmark corpora demonstrate the method's robustness, scalability, and adaptability, underscoring its value as a more holistic and dynamic alternative to conventional evaluation strategies.