# Evaluating Negative Sampling Approaches for Neural Topic Models

arXiv: https://arxiv.org/abs/2503.18167 · HTML: https://mlsanity.com/paper/2503.18167/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 45 references that are in the map.
- First posted on arXiv 23 Mar 2025.

## Builds on (most cited first, 25 of 25)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Topic Modeling in Embedding Spaces](https://mlsanity.com/paper/1907.04907/) (2019)
- [Autoencoding Variational Inference For Topic Models](https://mlsanity.com/paper/1703.01488/) (2017)
- [Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherence](https://mlsanity.com/paper/2004.03974/) (2020)
- [Neural Variational Inference for Text Processing](https://mlsanity.com/paper/1511.06038/) (2015)
- [Discovering Discrete Latent Topics with Neural Variational Inference](https://mlsanity.com/paper/1706.00359/) (2017)
- [Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherence](https://mlsanity.com/paper/2107.02173/) (2021)
- [Cross-lingual Contextualized Topic Models with Zero-shot Learning](https://mlsanity.com/paper/2004.07737/) (2020)
- [Contrastive Learning for Neural Topic Model](https://mlsanity.com/paper/2110.12764/) (2021)
- [Neural Models for Documents with Metadata](https://mlsanity.com/paper/1705.09296/) (2017)
- [Short Text Topic Modeling Techniques, Applications, and Performance: A Survey](https://mlsanity.com/paper/1904.07695/) (2019)
- [Neural Topic Modeling with Bidirectional Adversarial Training](https://mlsanity.com/paper/2004.12331/) (2020)
- [Prompting Large Language Models for Topic Modeling](https://mlsanity.com/paper/2312.09693/) (2023)
- [ATM:Adversarial-neural Topic Model](https://mlsanity.com/paper/1811.00265/) (2018)
- [Negative Sampling for Contrastive Representation Learning: A Review](https://mlsanity.com/paper/2206.00212/) (2022)
- [Does Negative Sampling Matter? A Review with Insights into its Theory and Applications](https://mlsanity.com/paper/2402.17238/) (2024)
- [Neural Topic Modeling with Cycle-Consistent Adversarial Training](https://mlsanity.com/paper/2009.13971/) (2020)
- [GPTopic: Dynamic and Interactive Topic Representations](https://mlsanity.com/paper/2403.03628/) (2024)
- [Improving Contextualized Topic Models with Negative Sampling](https://mlsanity.com/paper/2303.14951/) (2023)
- [GINopic: Topic Modeling with Graph Isomorphism Network](https://mlsanity.com/paper/2404.02115/) (2024)
- [Do Neural Topic Models Really Need Dropout? Analysis of the Effect of Dropout in Topic Modeling](https://mlsanity.com/paper/2303.15973/) (2023)

## Abstract (arXiv)

Negative sampling has emerged as an effective technique that enables deep learning models to learn better representations by introducing the paradigm of learn-to-compare. The goal of this approach is to add robustness to deep learning models to learn better representation by comparing the positive samples against the negative ones. Despite its numerous demonstrations in various areas of computer vision and natural language processing, a comprehensive study of the effect of negative sampling in an unsupervised domain like topic modeling has not been well explored. In this paper, we present a comprehensive analysis of the impact of different negative sampling strategies on neural topic models. We compare the performance of several popular neural topic models by incorporating a negative sampling technique in the decoder of variational autoencoder-based neural topic models. Experiments on four publicly available datasets demonstrate that integrating negative sampling into topic models results in significant enhancements across multiple aspects, including improved topic coherence, richer topic diversity, and more accurate document classification. Manual evaluations also indicate that the inclusion of negative sampling into neural topic models enhances the quality of the generated topics. These findings highlight the potential of negative sampling as a valuable tool for advancing the effectiveness of neural topic models.