# Efficient Vector Representation for Documents through Corruption

arXiv: https://arxiv.org/abs/1707.02377 · HTML: https://mlsanity.com/paper/1707.02377/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jul 2017.

## Built on by (most cited first, 16 of 16)

- [LongT5: Efficient Text-To-Text Transformer for Long Sequences](https://mlsanity.com/paper/2112.07916/) (2021)
- [KHAN: Knowledge-Aware Hierarchical Attention Networks for Accurate Political Stance Prediction](https://mlsanity.com/paper/2302.12126/) (2023)
- [Unsupervised Document Embedding via Contrastive Augmentation](https://mlsanity.com/paper/2103.14542/) (2021)
- [PAUSE: Positive and Annealed Unlabeled Sentence Embedding](https://mlsanity.com/paper/2109.03155/) (2021)
- [Retrofitting Multilingual Sentence Embeddings with Abstract Meaning Representation](https://mlsanity.com/paper/2210.09773/) (2022)
- [Unleashing the Power of LLMs in Dense Retrieval with Query Likelihood Modeling](https://mlsanity.com/paper/2504.05216/) (2025)
- [Unsupervised Sentence-embeddings by Manifold Approximation and Projection](https://mlsanity.com/paper/2102.03795/) (2021)
- [LAWDR: Language-Agnostic Weighted Document Representations from Pre-trained Models](https://mlsanity.com/paper/2106.03379/) (2021)
- [Dial2vec: Self-Guided Contrastive Learning of Unsupervised Dialogue Embeddings](https://mlsanity.com/paper/2210.15332/) (2022)
- [Parameter-Efficient Neural Question Answering Models via Graph-Enriched Document Representations](https://mlsanity.com/paper/2106.00851/) (2021)
- [Measuring a Texts Fairness Dimensions Using Machine Learning Based on Social Psychological Factors](https://mlsanity.com/paper/2111.00086/) (2021)
- [Investigation of Topic Modelling Methods for Understanding the Reports of the Mining Projects in Queensland](https://mlsanity.com/paper/2111.03576/) (2021)
- [Coherence-Based Distributed Document Representation Learning for Scientific Documents](https://mlsanity.com/paper/2201.02846/) (2022)
- [Improving Trace Link Recommendation by Using Non-Isotropic Distances and Combinations](https://mlsanity.com/paper/2307.07781/) (2023)
- [Learning Unsupervised Semantic Document Representation for Fine-grained Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2401.06210/) (2024)
- [Experiments with truth using Machine Learning: Spectral analysis and explainable classification of synthetic, false, and genuine information](https://mlsanity.com/paper/2407.05464/) (2024)

## Abstract (arXiv)

We present an efficient document representation learning framework, Document Vector through Corruption (Doc2VecC). Doc2VecC represents each document as a simple average of word embeddings. It ensures a representation generated as such captures the semantic meanings of the document during learning. A corruption model is included, which introduces a data-dependent regularization that favors informative or rare words while forcing the embeddings of common and non-discriminative ones to be close to zero. Doc2VecC produces significantly better word embeddings than Word2Vec. We compare Doc2VecC with several state-of-the-art document representation learning algorithms. The simple model architecture introduced by Doc2VecC matches or out-performs the state-of-the-art in generating high-quality document representations for sentiment analysis, document classification as well as semantic relatedness tasks. The simplicity of the model enables training on billions of words per hour on a single machine. At the same time, the model is very efficient in generating representations of unseen documents at test time.