# Learning document embeddings along with their uncertainties

arXiv: https://arxiv.org/abs/1908.07599 · HTML: https://mlsanity.com/paper/1908.07599/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Aug 2019.

## Built on by (most cited first, 4 of 4)

- [CAVEN: An Embodied Conversational Agent for Efficient Audio-Visual Navigation in Noisy Environments](https://mlsanity.com/paper/2306.04047/) (2023)
- [Sentiment Analysis of the COVID-related r/Depression Posts](https://mlsanity.com/paper/2108.06215/) (2021)
- [Performance of Data Augmentation Methods for Brazilian Portuguese Text Classification](https://mlsanity.com/paper/2304.02785/) (2023)
- [Thesis proposal: Are We Losing Textual Diversity to Natural Language Processing?](https://mlsanity.com/paper/2409.09568/) (2024)

## Abstract (arXiv)

Majority of the text modelling techniques yield only point-estimates of document embeddings and lack in capturing the uncertainty of the estimates. These uncertainties give a notion of how well the embeddings represent a document. We present Bayesian subspace multinomial model (Bayesian SMM), a generative log-linear model that learns to represent documents in the form of Gaussian distributions, thereby encoding the uncertainty in its co-variance. Additionally, in the proposed Bayesian SMM, we address a commonly encountered problem of intractability that appears during variational inference in mixed-logit models. We also present a generative Gaussian linear classifier for topic identification that exploits the uncertainty in document embeddings. Our intrinsic evaluation using perplexity measure shows that the proposed Bayesian SMM fits the data better as compared to the state-of-the-art neural variational document model on Fisher speech and 20Newsgroups text corpora. Our topic identification experiments show that the proposed systems are robust to over-fitting on unseen test data. The topic ID results show that the proposed model is outperforms state-of-the-art unsupervised topic models and achieve comparable results to the state-of-the-art fully supervised discriminative models.