# Improving the TENOR of Labeling: Re-evaluating Topic Models for Content Analysis

arXiv: https://arxiv.org/abs/2401.16348 · HTML: https://mlsanity.com/paper/2401.16348/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jan 2024.

## Built on by (most cited first, 10 of 10)

- [SciDoc2Diagrammer-MAF: Towards Generation of Scientific Diagrams from Documents guided by Multi-Aspect Feedback Refinement](https://mlsanity.com/paper/2409.19242/) (2024)
- [Teaching-Assistant-in-the-Loop: Improving Knowledge Distillation from Imperfect Teacher Models in Low-Budget Scenarios](https://mlsanity.com/paper/2406.05322/) (2024)
- [Multi-Stage Balanced Distillation: Addressing Long-Tail Challenges in Sequence-Level Knowledge Distillation](https://mlsanity.com/paper/2406.13114/) (2024)
- [ProxAnn: Use-Oriented Evaluations of Topic Models and Document Clustering](https://mlsanity.com/paper/2507.00828/) (2025)
- [Compositional Coordination for Multi-Robot Teams with Large Language Models](https://mlsanity.com/paper/2507.16068/) (2025)
- [Large Language Models Struggle to Describe the Haystack without Human Help: Human-in-the-loop Evaluation of Topic Models](https://mlsanity.com/paper/2502.14748/) (2025)
- [Large Language Models Are Effective Human Annotation Assistants, But Not Good Independent Annotators](https://mlsanity.com/paper/2503.06778/) (2025)
- [LLM-Driven Reasoning for Constraint-Aware Feature Selection in Industrial Systems](https://mlsanity.com/paper/2603.24979/) (2026)
- [Does Geo-co-location Matter? A Case Study of Public Health Conversations during COVID-19](https://mlsanity.com/paper/2405.17710/) (2024)
- [The Effect of Document Selection on Query-focused Text Analysis](https://mlsanity.com/paper/2604.12099/) (2026)

## Abstract (arXiv)

Topic models are a popular tool for understanding text collections, but their evaluation has been a point of contention. Automated evaluation metrics such as coherence are often used, however, their validity has been questioned for neural topic models (NTMs) and can overlook a models benefits in real world applications. To this end, we conduct the first evaluation of neural, supervised and classical topic models in an interactive task based setting. We combine topic models with a classifier and test their ability to help humans conduct content analysis and document annotation. From simulated, real user and expert pilot studies, the Contextual Neural Topic Model does the best on cluster evaluation metrics and human evaluations; however, LDA is competitive with two other NTMs under our simulated experiment and user study results, contrary to what coherence scores suggest. We show that current automated metrics do not provide a complete picture of topic modeling capabilities, but the right choice of NTMs can be better than classical models on practical task.