# Mixing Dirichlet Topic Models and Word Embeddings to Make lda2vec

arXiv: https://arxiv.org/abs/1605.02019 · HTML: https://mlsanity.com/paper/1605.02019/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 May 2016.

## Built on by (most cited first, 19 of 19)

- [Representing Mixtures of Word Embeddings with Mixtures of Topic Embeddings](https://mlsanity.com/paper/2203.01570/) (2022)
- [Searching for Structure in Unfalsifiable Claims](https://mlsanity.com/paper/2209.00495/) (2022)
- [Topics as Entity Clusters: Entity-based Topics from Large Language Models and Graph Neural Networks](https://mlsanity.com/paper/2301.02458/) (2023)
- [Causal effect of racial bias in data and machine learning algorithms on user persuasiveness & discriminatory decision making: An Empirical Study](https://mlsanity.com/paper/2202.00471/) (2022)
- [Automated Code Extraction from Discussion Board Text Dataset](https://mlsanity.com/paper/2210.17495/) (2022)
- [Exploring the Power of Topic Modeling Techniques in Analyzing Customer Reviews: A Comparative Analysis](https://mlsanity.com/paper/2308.11520/) (2023)
- [Network representation learning systematic review: ancestors and current development state](https://mlsanity.com/paper/2109.07583/) (2021)
- [Survey of Generative Methods for Social Media Analysis](https://mlsanity.com/paper/2112.07041/) (2021)
- [TSM: Measuring the Enticement of Honeyfiles with Natural Language Processing](https://mlsanity.com/paper/2203.07580/) (2022)
- [Exploring the evolution of research topics during the COVID-19 pandemic](https://mlsanity.com/paper/2310.03928/) (2023)
- [Capturing research literature attitude towards Sustainable Development Goals: an LLM-based topic modeling approach](https://mlsanity.com/paper/2411.02943/) (2024)
- [Deep Probabilistic Graphical Modeling](https://mlsanity.com/paper/2104.12053/) (2021)
- [Semiparametric Latent Topic Modeling on Consumer-Generated Corpora](https://mlsanity.com/paper/2107.10651/) (2021)
- [Graph2topic: an opensource topic modeling framework based on sentence embedding and community detection](https://mlsanity.com/paper/2304.06653/) (2023)
- [A Process for Topic Modelling Via Word Embeddings](https://mlsanity.com/paper/2312.03705/) (2023)
- [A Novel Method for News Article Event-Based Embedding](https://mlsanity.com/paper/2405.13071/) (2024)
- [Normalisation of SWIFT Message Counterparties with Feature Extraction and Clustering](https://mlsanity.com/paper/2508.21081/) (2025)
- [ProtoTopic: Prototypical Network for Few-Shot Medical Topic Modeling](https://mlsanity.com/paper/2510.13542/) (2025)
- [Automating Historical Insight Extraction from Large-Scale Newspaper Archives via Neural Topic Modeling](https://mlsanity.com/paper/2512.11635/) (2025)

## Abstract (arXiv)

Distributed dense word vectors have been shown to be effective at capturing token-level semantic and syntactic regularities in language, while topic models can form interpretable representations over documents. In this work, we describe lda2vec, a model that learns dense word vectors jointly with Dirichlet-distributed latent document-level mixtures of topic vectors. In contrast to continuous dense document representations, this formulation produces sparse, interpretable document mixtures through a non-negative simplex constraint. Our method is simple to incorporate into existing automatic differentiation frameworks and allows for unsupervised document representations geared for use by scientists while simultaneously learning word vectors and the linear relationships between them.