# Topic Modeling with Wasserstein Autoencoders

arXiv: https://arxiv.org/abs/1907.12374 · HTML: https://mlsanity.com/paper/1907.12374/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jul 2019.

## Built on by (most cited first, 30 of 53)

- [Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherence](https://mlsanity.com/paper/2107.02173/) (2021)
- [Topic Modelling Meets Deep Neural Networks: A Survey](https://mlsanity.com/paper/2103.00498/) (2021)
- [Contrastive Learning for Neural Topic Model](https://mlsanity.com/paper/2110.12764/) (2021)
- [Effective Neural Topic Modeling with Embedding Clustering Regularization](https://mlsanity.com/paper/2306.04217/) (2023)
- [Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learning](https://mlsanity.com/paper/2211.12878/) (2022)
- [Representing Mixtures of Word Embeddings with Mixtures of Topic Embeddings](https://mlsanity.com/paper/2203.01570/) (2022)
- [Is Neural Topic Modelling Better than Clustering? An Empirical Study on Clustering with Contextual Embeddings for Topics](https://mlsanity.com/paper/2204.09874/) (2022)
- [InfoCTM: A Mutual Information Maximization Perspective of Cross-Lingual Topic Modeling](https://mlsanity.com/paper/2304.03544/) (2023)
- [Sawtooth Factorial Topic Embeddings Guided Gamma Belief Network](https://mlsanity.com/paper/2107.02757/) (2021)
- [Towards the TopMost: A Topic Modeling System Toolkit](https://mlsanity.com/paper/2309.06908/) (2023)
- [A Survey on Neural Topic Models: Methods, Applications, and Challenges](https://mlsanity.com/paper/2401.15351/) (2024)
- [HyperMiner: Topic Taxonomy Mining with Hyperbolic Embedding](https://mlsanity.com/paper/2210.10625/) (2022)
- [Neural Topic Modeling of Psychotherapy Sessions](https://mlsanity.com/paper/2204.10189/) (2022)
- [Topic Modeling as Multi-Objective Contrastive Optimization](https://mlsanity.com/paper/2402.07577/) (2024)
- [Event Temporal Relation Extraction with Bayesian Translational Model](https://mlsanity.com/paper/2302.04985/) (2023)
- [SupervisorBot: NLP-Annotated Real-Time Recommendations of Psychotherapy Treatment Strategies with Deep Reinforcement Learning](https://mlsanity.com/paper/2208.13077/) (2022)
- [COMPASS: Computational Mapping of Patient-Therapist Alliance Strategies with Language Modeling](https://mlsanity.com/paper/2402.14701/) (2024)
- [Knowledge Management System with NLP-Assisted Annotations: A Brief Survey and Outlook](https://mlsanity.com/paper/2206.07304/) (2022)
- [CWTM: Leveraging Contextualized Word Embeddings from BERT for Neural Topic Modeling](https://mlsanity.com/paper/2305.09329/) (2023)
- [Neural Attention-Aware Hierarchical Topic Model](https://mlsanity.com/paper/2110.07161/) (2021)
- [A Topic-aware Summarization Framework with Different Modal Side Information](https://mlsanity.com/paper/2305.11503/) (2023)
- [Diversity-Aware Coherence Loss for Improving Neural Topic Models](https://mlsanity.com/paper/2305.16199/) (2023)
- [Topic-VQ-VAE: Leveraging Latent Codebooks for Flexible Topic-Guided Document Generation](https://mlsanity.com/paper/2312.11532/) (2023)
- [Enhancing Short-Text Topic Modeling with LLM-Driven Context Expansion and Prefix-Tuned VAEs](https://mlsanity.com/paper/2410.03071/) (2024)
- [Self-supervised Topic Taxonomy Discovery in the Box Embedding Space](https://mlsanity.com/paper/2408.15050/) (2024)
- [Bridging the Evaluation Gap: Leveraging Large Language Models for Topic Model Evaluation](https://mlsanity.com/paper/2502.07352/) (2025)
- [Evaluation of Thematic Coherence in Microblogs](https://mlsanity.com/paper/2106.15971/) (2021)
- [A Joint Learning Approach for Semi-supervised Neural Topic Modeling](https://mlsanity.com/paper/2204.03208/) (2022)
- [TherapyView: Visualizing Therapy Sessions with Temporal Topic Modeling and AI-Generated Arts](https://mlsanity.com/paper/2302.10845/) (2023)
- [CUE: An Uncertainty Interpretation Framework for Text Classifiers Built on Pre-Trained Language Models](https://mlsanity.com/paper/2306.03598/) (2023)

## Abstract (arXiv)

We propose a novel neural topic model in the Wasserstein autoencoders (WAE) framework. Unlike existing variational autoencoder based models, we directly enforce Dirichlet prior on the latent document-topic vectors. We exploit the structure of the latent space and apply a suitable kernel in minimizing the Maximum Mean Discrepancy (MMD) to perform distribution matching. We discover that MMD performs much better than the Generative Adversarial Network (GAN) in matching high dimensional Dirichlet distribution. We further discover that incorporating randomness in the encoder output during training leads to significantly more coherent topics. To measure the diversity of the produced topics, we propose a simple topic uniqueness metric. Together with the widely used coherence measure NPMI, we offer a more wholistic evaluation of topic quality. Experiments on several real datasets show that our model produces significantly better topics than existing topic models.