# Word Mover's Embedding: From Word2Vec to Document Embedding

arXiv: https://arxiv.org/abs/1811.01713 · HTML: https://mlsanity.com/paper/1811.01713/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2018.

## Built on by (most cited first, 26 of 26)

- [LongT5: Efficient Text-To-Text Transformer for Long Sequences](https://mlsanity.com/paper/2112.07916/) (2021)
- [SciRepEval: A Multi-Format Benchmark for Scientific Document Representations](https://mlsanity.com/paper/2211.13308/) (2022)
- [Toward Interpretable Semantic Textual Similarity via Optimal Transport-based Contrastive Sentence Learning](https://mlsanity.com/paper/2202.13196/) (2022)
- [Sentence Similarity Based on Contexts](https://mlsanity.com/paper/2105.07623/) (2021)
- [Re-evaluating Word Mover's Distance](https://mlsanity.com/paper/2105.14403/) (2021)
- [Deep Cut-informed Graph Embedding and Clustering](https://mlsanity.com/paper/2503.06635/) (2025)
- [Plug-and-Play Document Modules for Pre-trained Models](https://mlsanity.com/paper/2305.17660/) (2023)
- [SlideSpawn: An Automatic Slides Generation System for Research Publications](https://mlsanity.com/paper/2411.17719/) (2024)
- [Unsupervised Document Embedding via Contrastive Augmentation](https://mlsanity.com/paper/2103.14542/) (2021)
- [PAUSE: Positive and Annealed Unlabeled Sentence Embedding](https://mlsanity.com/paper/2109.03155/) (2021)
- [Automatic Controllable Product Copywriting for E-Commerce](https://mlsanity.com/paper/2206.10103/) (2022)
- [SecureReviewer: Enhancing Large Language Models for Secure Code Review through Secure-aware Fine-tuning](https://mlsanity.com/paper/2510.26457/) (2025)
- [Unsupervised Sentence-embeddings by Manifold Approximation and Projection](https://mlsanity.com/paper/2102.03795/) (2021)
- [Order Constraints in Optimal Transport](https://mlsanity.com/paper/2110.07275/) (2021)
- [Unsupervised Sentence Textual Similarity with Compositional Phrase Semantics](https://mlsanity.com/paper/2210.02284/) (2022)
- [Sublinear Time Approximation of Text Similarity Matrices](https://mlsanity.com/paper/2112.09631/) (2021)
- [Efficient approximation of Earth Mover's Distance Based on Nearest Neighbor Search](https://mlsanity.com/paper/2401.07378/) (2024)
- [Constructing Contrastive samples via Summarization for Text Classification with limited annotations](https://mlsanity.com/paper/2104.05094/) (2021)
- [Quantifying the Dissimilarity of Texts](https://mlsanity.com/paper/2305.02457/) (2023)
- [A New Parallel Algorithm for Sinkhorn Word-Movers Distance and Its Performance on PIUMA and Xeon CPU](https://mlsanity.com/paper/2107.06433/) (2021)
- [Building a Question Answering System for the Manufacturing Domain](https://mlsanity.com/paper/2111.10044/) (2021)
- [Coherence-Based Distributed Document Representation Learning for Scientific Documents](https://mlsanity.com/paper/2201.02846/) (2022)
- [Cross-Media Scientific Research Achievements Retrieval Based on Deep Language Model](https://mlsanity.com/paper/2203.15595/) (2022)
- [Decoding the Alphabet Soup of Degrees in the United States Postsecondary Education System Through Hybrid Method: Database and Text Mining](https://mlsanity.com/paper/2309.13050/) (2023)
- [Learning Unsupervised Semantic Document Representation for Fine-grained Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2401.06210/) (2024)
- [X-ray transferable polyrepresentation learning](https://mlsanity.com/paper/2507.06264/) (2025)

## Abstract (arXiv)

While the celebrated Word2Vec technique yields semantically rich representations for individual words, there has been relatively less success in extending to generate unsupervised sentences or documents embeddings. Recent work has demonstrated that a distance measure between documents called Word Mover's Distance (WMD) that aligns semantically similar words, yields unprecedented KNN classification accuracy. However, WMD is expensive to compute, and it is hard to extend its use beyond a KNN classifier. In this paper, we propose the Word Mover's Embedding  (WME), a novel approach to building an unsupervised document (sentence) embedding from pre-trained word embeddings. In our experiments on 9 benchmark text classification datasets and 22 textual similarity tasks, the proposed technique consistently matches or outperforms state-of-the-art techniques, with significantly higher accuracy on problems of short length.