# Improving Contextualized Topic Models with Negative Sampling

arXiv: https://arxiv.org/abs/2303.14951 · HTML: https://mlsanity.com/paper/2303.14951/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 26 references that are in the map.
- First posted on arXiv 27 Mar 2023.

## Builds on (most cited first, 14 of 14)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Topic Modeling in Embedding Spaces](https://mlsanity.com/paper/1907.04907/) (2019)
- [Autoencoding Variational Inference For Topic Models](https://mlsanity.com/paper/1703.01488/) (2017)
- [Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherence](https://mlsanity.com/paper/2004.03974/) (2020)
- [Neural Variational Inference for Text Processing](https://mlsanity.com/paper/1511.06038/) (2015)
- [Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherence](https://mlsanity.com/paper/2107.02173/) (2021)
- [Topic Modelling Meets Deep Neural Networks: A Survey](https://mlsanity.com/paper/2103.00498/) (2021)
- [Contrastive Learning for Neural Topic Model](https://mlsanity.com/paper/2110.12764/) (2021)
- [Short Text Topic Modeling Techniques, Applications, and Performance: A Survey](https://mlsanity.com/paper/1904.07695/) (2019)
- [Neural Topic Modeling with Bidirectional Adversarial Training](https://mlsanity.com/paper/2004.12331/) (2020)
- [ATM:Adversarial-neural Topic Model](https://mlsanity.com/paper/1811.00265/) (2018)
- [Negative Sampling for Contrastive Representation Learning: A Review](https://mlsanity.com/paper/2206.00212/) (2022)
- [Neural Topic Modeling with Cycle-Consistent Adversarial Training](https://mlsanity.com/paper/2009.13971/) (2020)

## Built on by (most cited first, 4 of 4)

- [GINopic: Topic Modeling with Graph Isomorphism Network](https://mlsanity.com/paper/2404.02115/) (2024)
- [What are Public Concerns about ChatGPT? A Novel Self-Supervised Neural Topic Model Tells You](https://mlsanity.com/paper/2309.01522/) (2023)
- [Evaluating Negative Sampling Approaches for Neural Topic Models](https://mlsanity.com/paper/2503.18167/) (2025)
- [Human-Centric Topic Modeling with Goal-Prompted Contrastive Learning and Optimal Transport](https://mlsanity.com/paper/2604.12663/) (2026)

## Abstract (arXiv)

Topic modeling has emerged as a dominant method for exploring large document collections. Recent approaches to topic modeling use large contextualized language models and variational autoencoders. In this paper, we propose a negative sampling mechanism for a contextualized topic model to improve the quality of the generated topics. In particular, during model training, we perturb the generated document-topic vector and use a triplet loss to encourage the document reconstructed from the correct document-topic vector to be similar to the input document and dissimilar to the document reconstructed from the perturbed vector. Experiments for different topic counts on three publicly available benchmark datasets show that in most cases, our approach leads to an increase in topic coherence over that of the baselines. Our model also achieves very high topic diversity.