# Neural Topic Modeling with Large Language Models in the Loop

arXiv: https://arxiv.org/abs/2411.08534 · HTML: https://mlsanity.com/paper/2411.08534/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 94 references that are in the map.
- First posted on arXiv 13 Nov 2024.

## Builds on (most cited first, 30 of 44)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback](https://mlsanity.com/paper/2305.14975/) (2023)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Yi: Open Foundation Models by 01.AI](https://mlsanity.com/paper/2403.04652/) (2024)
- [Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution](https://mlsanity.com/paper/2309.16797/) (2023)
- [A Survey of Confidence Estimation and Calibration in Large Language Models](https://mlsanity.com/paper/2311.08298/) (2023)
- [Topic Modeling in Embedding Spaces](https://mlsanity.com/paper/1907.04907/) (2019)
- [Out-of-Distribution Detection and Selective Generation for Conditional Language Models](https://mlsanity.com/paper/2209.15558/) (2022)
- [Learning with a Wasserstein Loss](https://mlsanity.com/paper/1506.05439/) (2015)
- [OTA: Optimal Transport Assignment for Object Detection](https://mlsanity.com/paper/2103.14259/) (2021)
- [Autoencoding Variational Inference For Topic Models](https://mlsanity.com/paper/1703.01488/) (2017)
- [Large-Scale Optimal Transport and Mapping Estimation](https://mlsanity.com/paper/1711.02283/) (2017)
- [Quantifying Uncertainty in Answers from any Language Model and Enhancing their Trustworthiness](https://mlsanity.com/paper/2308.16175/) (2023)
- [Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherence](https://mlsanity.com/paper/2004.03974/) (2020)
- [Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling](https://mlsanity.com/paper/2311.08718/) (2023)
- [Discovering Discrete Latent Topics with Neural Variational Inference](https://mlsanity.com/paper/1706.00359/) (2017)
- [Tired of Topic Models? Clusters of Pretrained Word Embeddings Make for Fast and Good Topics too!](https://mlsanity.com/paper/2004.14914/) (2020)
- [Calibrating Large Language Models Using Their Generations Only](https://mlsanity.com/paper/2403.05973/) (2024)
- [Topic Modelling Meets Deep Neural Networks: A Survey](https://mlsanity.com/paper/2103.00498/) (2021)
- [Neural Topic Model via Optimal Transport](https://mlsanity.com/paper/2008.13537/) (2020)
- [Contrastive Learning for Neural Topic Model](https://mlsanity.com/paper/2110.12764/) (2021)
- [Neural Models for Documents with Metadata](https://mlsanity.com/paper/1705.09296/) (2017)
- [Effective Neural Topic Modeling with Embedding Clustering Regularization](https://mlsanity.com/paper/2306.04217/) (2023)
- [A Unified Wasserstein Distributional Robustness Framework for Adversarial Training](https://mlsanity.com/paper/2202.13437/) (2022)

## Built on by (most cited first, 10 of 10)

- [Quantifying Fairness in LLMs Beyond Tokens: A Semantic and Statistical Perspective](https://mlsanity.com/paper/2506.19028/) (2025)
- [Next Generation Active Learning: Mixture of LLMs in the Loop](https://mlsanity.com/paper/2601.15773/) (2026)
- [SciTopic: Enhancing Topic Discovery in Scientific Literature through Advanced LLM](https://mlsanity.com/paper/2508.20514/) (2025)
- [LLM-XTM: Enhancing Cross-Lingual Topic Models with Large Language Models](https://mlsanity.com/paper/2605.03299/) (2026)
- [Generating Natural-Language Surgical Feedback: From Structured Representation to Domain-Grounded Evaluation](https://mlsanity.com/paper/2511.15159/) (2025)
- [Improving Topic Modeling by Distilling Soft Labels from Language Models](https://mlsanity.com/paper/2602.17907/) (2026)
- [A Neural Topic Method Using a Large-Language-Model-in-the-Loop for Business Research](https://mlsanity.com/paper/2603.03623/) (2026)
- [Human-Centric Topic Modeling with Goal-Prompted Contrastive Learning and Optimal Transport](https://mlsanity.com/paper/2604.12663/) (2026)
- [How Much Structure Do LLMs Need? Evaluating LLMs for Bibliometric Cluster Description](https://mlsanity.com/paper/2605.24351/) (2026)
- [Segment-Level Agentic Topic Modeling for Improved Data Exploration and Resource Efficiency](https://mlsanity.com/paper/2609.31460/) (2026)

## Abstract (arXiv)

Topic modeling is a fundamental task in natural language processing, allowing the discovery of latent thematic structures in text corpora. While Large Language Models (LLMs) have demonstrated promising capabilities in topic discovery, their direct application to topic modeling suffers from issues such as incomplete topic coverage, misalignment of topics, and inefficiency. To address these limitations, we propose LLM-ITL, a novel LLM-in-the-loop framework that integrates LLMs with Neural Topic Models (NTMs). In LLM-ITL, global topics and document representations are learned through the NTM. Meanwhile, an LLM refines these topics using an Optimal Transport (OT)-based alignment objective, where the refinement is dynamically adjusted based on the LLM's confidence in suggesting topical words for each set of input words. With the flexibility of being integrated into many existing NTMs, the proposed approach enhances the interpretability of topics while preserving the efficiency of NTMs in learning topics and document representations. Extensive experiments demonstrate that LLM-ITL helps NTMs significantly improve their topic interpretability while maintaining the quality of document representation. Our code and datasets are available at https://github.com/Xiaohao-Yang/LLM-ITL