# Distributed Representations of Sentences and Documents

arXiv: https://arxiv.org/abs/1405.4053 · HTML: https://mlsanity.com/paper/1405.4053/

## Key facts

- Cited by 678 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 May 2014.

## Built on by (most cited first, 30 of 678)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GLaM: Efficient Scaling of Language Models with Mixture-of-Experts](https://mlsanity.com/paper/2112.06905/) (2021)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Generating Datasets with Pretrained Language Models](https://mlsanity.com/paper/2104.07540/) (2021)
- [Text Embeddings Reveal (Almost) As Much As Text](https://mlsanity.com/paper/2310.06816/) (2023)
- [MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrieval](https://mlsanity.com/paper/2307.00589/) (2023)
- [Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond](https://mlsanity.com/paper/2109.00725/) (2021)
- [SciRepEval: A Multi-Format Benchmark for Scientific Document Representations](https://mlsanity.com/paper/2211.13308/) (2022)
- [Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network?](https://mlsanity.com/paper/2203.09081/) (2022)
- [Gemini Embedding: Generalizable Embeddings from Gemini](https://mlsanity.com/paper/2503.07891/) (2025)
- [When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunities](https://mlsanity.com/paper/2307.16376/) (2023)
- [A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges](https://mlsanity.com/paper/2406.11903/) (2024)
- [Gecko: Versatile Text Embeddings Distilled from Large Language Models](https://mlsanity.com/paper/2403.20327/) (2024)
- [ILDC for CJPE: Indian Legal Documents Corpus for Court Judgment Prediction and Explanation](https://mlsanity.com/paper/2105.13562/) (2021)
- [PEER: A Comprehensive and Multi-Task Benchmark for Protein Sequence Understanding](https://mlsanity.com/paper/2206.02096/) (2022)
- [Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey](https://mlsanity.com/paper/2411.17558/) (2024)
- [ESimCSE: Enhanced Sample Building Method for Contrastive Learning of Unsupervised Sentence Embedding](https://mlsanity.com/paper/2109.04380/) (2021)
- [Time-MMD: Multi-Domain Multimodal Dataset for Time Series Analysis](https://mlsanity.com/paper/2406.08627/) (2024)
- [A Comprehensive Survey on Deep Clustering: Taxonomy, Challenges, and Future Directions](https://mlsanity.com/paper/2206.07579/) (2022)
- [Repetition Improves Language Model Embeddings](https://mlsanity.com/paper/2402.15449/) (2024)
- [MDFEND: Multi-domain Fake News Detection](https://mlsanity.com/paper/2201.00987/) (2022)
- [Twin Contrastive Learning for Online Clustering](https://mlsanity.com/paper/2210.11680/) (2022)
- [Log-based Anomaly Detection Without Log Parsing](https://mlsanity.com/paper/2108.01955/) (2021)
- [The Wall Street Neophyte: A Zero-Shot Analysis of ChatGPT Over MultiModal Stock Movement Prediction Challenges](https://mlsanity.com/paper/2304.05351/) (2023)
- [Debiased Contrastive Learning of Unsupervised Sentence Representations](https://mlsanity.com/paper/2205.00656/) (2022)
- [Gloss Attention for Gloss-free Sign Language Translation](https://mlsanity.com/paper/2307.07361/) (2023)
- [PalmTree: Learning an Assembly Language Model for Instruction Embedding](https://mlsanity.com/paper/2103.03809/) (2021)
- [Text2Earth: Unlocking Text-driven Remote Sensing Image Generation with a Global-Scale Dataset and a Foundation Model](https://mlsanity.com/paper/2501.00895/) (2025)
- [A Survey on Ensemble Learning under the Era of Deep Learning](https://mlsanity.com/paper/2101.08387/) (2021)
- [Evaluating Unsupervised Text Classification: Zero-shot and Similarity-based Approaches](https://mlsanity.com/paper/2211.16285/) (2022)

## Abstract (arXiv)

Many machine learning algorithms require the input to be represented as a fixed-length feature vector. When it comes to texts, one of the most common fixed-length features is bag-of-words. Despite their popularity, bag-of-words features have two major weaknesses: they lose the ordering of the words and they also ignore semantics of the words. For example, "powerful," "strong" and "Paris" are equally distant. In this paper, we propose Paragraph Vector, an unsupervised algorithm that learns fixed-length feature representations from variable-length pieces of texts, such as sentences, paragraphs, and documents. Our algorithm represents each document by a dense vector which is trained to predict words in the document. Its construction gives our algorithm the potential to overcome the weaknesses of bag-of-words models. Empirical results show that Paragraph Vectors outperform bag-of-words models as well as other techniques for text representations. Finally, we achieve new state-of-the-art results on several text classification and sentiment analysis tasks.