# Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features

arXiv: https://arxiv.org/abs/1703.02507 · HTML: https://mlsanity.com/paper/1703.02507/

## Key facts

- Cited by 104 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Mar 2017.

## Built on by (most cited first, 30 of 104)

- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrieval](https://mlsanity.com/paper/2307.00589/) (2023)
- [TSDAE: Using Transformer-based Sequential Denoising Auto-Encoder for Unsupervised Sentence Embedding Learning](https://mlsanity.com/paper/2104.06979/) (2021)
- [AnglE-optimized Text Embeddings](https://mlsanity.com/paper/2309.12871/) (2023)
- [Unsupervised Time-Series Representation Learning with Iterative Bilinear Temporal-Spectral Fusion](https://mlsanity.com/paper/2202.04770/) (2022)
- [ILDC for CJPE: Indian Legal Documents Corpus for Court Judgment Prediction and Explanation](https://mlsanity.com/paper/2105.13562/) (2021)
- [Elaborative Rehearsal for Zero-shot Action Recognition](https://mlsanity.com/paper/2108.02833/) (2021)
- [ESimCSE: Enhanced Sample Building Method for Contrastive Learning of Unsupervised Sentence Embedding](https://mlsanity.com/paper/2109.04380/) (2021)
- [TimesURL: Self-supervised Contrastive Learning for Universal Time Series Representation Learning](https://mlsanity.com/paper/2312.15709/) (2023)
- [N-Gram in Swin Transformers for Efficient Lightweight Image Super-Resolution](https://mlsanity.com/paper/2211.11436/) (2022)
- [Multimodal Entity Linking for Tweets](https://mlsanity.com/paper/2104.03236/) (2021)
- [Learn2Augment: Learning to Composite Videos for Data Augmentation in Action Recognition](https://mlsanity.com/paper/2206.04790/) (2022)
- [InfoCSE: Information-aggregated Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2210.06432/) (2022)
- [LeSICiN: A Heterogeneous Graph-based Approach for Automatic Legal Statute Identification from Indian Legal Documents](https://mlsanity.com/paper/2112.14731/) (2021)
- [Entity Linking Meets Deep Learning: Techniques and Solutions](https://mlsanity.com/paper/2109.12520/) (2021)
- [Paraphrastic Representations at Scale](https://mlsanity.com/paper/2104.15114/) (2021)
- [Norm of Word Embedding Encodes Information Gain](https://mlsanity.com/paper/2212.09663/) (2022)
- [MDERank: A Masked Document Embedding Rank Approach for Unsupervised Keyphrase Extraction](https://mlsanity.com/paper/2110.06651/) (2021)
- [Abstract, Rationale, Stance: A Joint Model for Scientific Claim Verification](https://mlsanity.com/paper/2110.15116/) (2021)
- [MemSum: Extractive Summarization of Long Documents Using Multi-Step Episodic Markov Decision Processes](https://mlsanity.com/paper/2107.08929/) (2021)
- [Local Citation Recommendation with Hierarchical-Attention Text Encoder and SciBERT-based Reranking](https://mlsanity.com/paper/2112.01206/) (2021)
- [PatternRank: Leveraging Pretrained Language Models and Part of Speech for Unsupervised Keyphrase Extraction](https://mlsanity.com/paper/2210.05245/) (2022)
- [Obtaining Better Static Word Embeddings Using Contextual Embedding Models](https://mlsanity.com/paper/2106.04302/) (2021)
- [Unsupervised Keyphrase Extraction by Jointly Modeling Local and Global Context](https://mlsanity.com/paper/2109.07293/) (2021)
- [One-shot action recognition in challenging therapy scenarios](https://mlsanity.com/paper/2102.08997/) (2021)
- [Cross-modal Representation Learning for Zero-shot Action Recognition](https://mlsanity.com/paper/2205.01657/) (2022)
- [SimCSE++: Improving Contrastive Learning for Sentence Embeddings from Two Perspectives](https://mlsanity.com/paper/2305.13192/) (2023)
- [A New Split for Evaluating True Zero-Shot Action Recognition](https://mlsanity.com/paper/2107.13029/) (2021)
- [From Statistical Methods to Deep Learning, Automatic Keyphrase Prediction: A Survey](https://mlsanity.com/paper/2305.02579/) (2023)
- [KDMCSE: Knowledge Distillation Multimodal Sentence Embeddings with Adaptive Angular margin Contrastive Learning](https://mlsanity.com/paper/2403.17486/) (2024)

## Abstract (arXiv)

The recent tremendous success of unsupervised word embeddings in a multitude of applications raises the obvious question if similar methods could be derived to improve embeddings (i.e. semantic representations) of word sequences as well. We present a simple but efficient unsupervised objective to train distributed representations of sentences. Our method outperforms the state-of-the-art unsupervised models on most benchmark tasks, highlighting the robustness of the produced general-purpose sentence embeddings.