# Diversity-Aware Coherence Loss for Improving Neural Topic Models

arXiv: https://arxiv.org/abs/2305.16199 · HTML: https://mlsanity.com/paper/2305.16199/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 40 references that are in the map.
- First posted on arXiv 25 May 2023.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Topic Modeling in Embedding Spaces](https://mlsanity.com/paper/1907.04907/) (2019)
- [Autoencoding Variational Inference For Topic Models](https://mlsanity.com/paper/1703.01488/) (2017)
- [Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherence](https://mlsanity.com/paper/2004.03974/) (2020)
- [Neural Variational Inference for Text Processing](https://mlsanity.com/paper/1511.06038/) (2015)
- [Discovering Discrete Latent Topics with Neural Variational Inference](https://mlsanity.com/paper/1706.00359/) (2017)
- [Cross-lingual Contextualized Topic Models with Zero-shot Learning](https://mlsanity.com/paper/2004.07737/) (2020)
- [Topic-Driven and Knowledge-Aware Transformer for Dialogue Emotion Detection](https://mlsanity.com/paper/2106.01071/) (2021)
- [Topic Modeling with Wasserstein Autoencoders](https://mlsanity.com/paper/1907.12374/) (2019)
- [Neural Models for Documents with Metadata](https://mlsanity.com/paper/1705.09296/) (2017)
- [Topic-Aware Multi-turn Dialogue Modeling](https://mlsanity.com/paper/2009.12539/) (2020)
- [Topic-Guided Variational Autoencoders for Text Generation](https://mlsanity.com/paper/1903.07137/) (2019)
- [Short Text Topic Modeling Techniques, Applications, and Performance: A Survey](https://mlsanity.com/paper/1904.07695/) (2019)
- [Coherence-Aware Neural Topic Modeling](https://mlsanity.com/paper/1809.02687/) (2018)
- [Improving Neural Topic Models using Knowledge Distillation](https://mlsanity.com/paper/2010.02377/) (2020)
- [Topic Segmentation and Labeling in Asynchronous Conversations](https://mlsanity.com/paper/1402.0586/) (2014)
- [Attend to the Right Context: A Plug-and-Play Module for Content-Controllable Summarization](https://mlsanity.com/paper/2212.10819/) (2022)
- [Evaluating Topic Quality with Posterior Variability](https://mlsanity.com/paper/1909.03524/) (2019)

## Built on by (most cited first, 5 of 5)

- [DeTiME: Diffusion-Enhanced Topic Modeling using Encoder-decoder based LLM](https://mlsanity.com/paper/2310.15296/) (2023)
- [Neural Multimodal Topic Modeling: A Comprehensive Evaluation](https://mlsanity.com/paper/2403.17308/) (2024)
- [A survey of diversity quantification in natural language processing: The why, what, where and how](https://mlsanity.com/paper/2507.20858/) (2025)
- [Joint Modeling of Entities and Discourse Relations for Coherence Assessment](https://mlsanity.com/paper/2509.04182/) (2025)
- [Improving Topic Modeling by Distilling Soft Labels from Language Models](https://mlsanity.com/paper/2602.17907/) (2026)

## Abstract (arXiv)

The standard approach for neural topic modeling uses a variational autoencoder (VAE) framework that jointly minimizes the KL divergence between the estimated posterior and prior, in addition to the reconstruction loss. Since neural topic models are trained by recreating individual input documents, they do not explicitly capture the coherence between topic words on the corpus level. In this work, we propose a novel diversity-aware coherence loss that encourages the model to learn corpus-level coherence scores while maintaining a high diversity between topics. Experimental results on multiple datasets show that our method significantly improves the performance of neural topic models without requiring any pretraining or additional parameters.