# Improving a tf-idf weighted document vector embedding

arXiv: https://arxiv.org/abs/1902.09875 · HTML: https://mlsanity.com/paper/1902.09875/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Feb 2019.

## Built on by (most cited first, 2 of 2)

- [Generative User-Experience Research for Developing Domain-specific Natural Language Processing Applications](https://mlsanity.com/paper/2306.16143/) (2023)
- [Coherence-Based Distributed Document Representation Learning for Scientific Documents](https://mlsanity.com/paper/2201.02846/) (2022)

## Abstract (arXiv)

We examine a number of methods to compute a dense vector embedding for a document in a corpus, given a set of word vectors such as those from word2vec or GloVe. We describe two methods that can improve upon a simple weighted sum, that are optimal in the sense that they maximizes a particular weighted cosine similarity measure. We consider several weighting functions, including inverse document frequency (idf), smooth inverse frequency (SIF), and the sub-sampling function used in word2vec. We find that idf works best for our applications. We also use common component removal proposed by Arora et al. as a post-process and find it is helpful in most cases. We compare these embeddings variations to the doc2vec embedding on a new evaluation task using TripAdvisor reviews, and also on the CQADupStack benchmark from the literature.