# EASE: Entity-Aware Contrastive Learning of Sentence Embedding

arXiv: https://arxiv.org/abs/2205.04260 · HTML: https://mlsanity.com/paper/2205.04260/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 62 references that are in the map.
- First posted on arXiv 9 May 2022.

## Builds on (most cited first, 30 of 35)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation](https://mlsanity.com/paper/1708.00055/) (2017)
- [XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization](https://mlsanity.com/paper/2003.11080/) (2020)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [Contrastive Learning with Hard Negative Samples](https://mlsanity.com/paper/2010.04592/) (2020)
- [Language-agnostic BERT Sentence Embedding](https://mlsanity.com/paper/2007.01852/) (2020)
- [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020)
- [Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond](https://mlsanity.com/paper/1812.10464/) (2018)
- [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017)
- [Skip-Thought Vectors](https://mlsanity.com/paper/1506.06726/) (2015)
- [DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations](https://mlsanity.com/paper/2006.03659/) (2020)
- [ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer](https://mlsanity.com/paper/2105.11741/) (2021)
- [An efficient framework for learning sentence representations](https://mlsanity.com/paper/1803.02893/) (2018)
- [CERT: Contrastive Self-supervised Learning for Language Understanding](https://mlsanity.com/paper/2005.12766/) (2020)
- [COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining](https://mlsanity.com/paper/2102.08473/) (2021)
- [Self-Guided Contrastive Learning for BERT Sentence Representations](https://mlsanity.com/paper/2106.07345/) (2021)
- [Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model](https://mlsanity.com/paper/1912.09637/) (2019)
- [Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders](https://mlsanity.com/paper/2104.08027/) (2021)
- [Learning Joint Multilingual Sentence Representations with Neural Machine Translation](https://mlsanity.com/paper/1704.04154/) (2017)
- [A Corpus for Multilingual Document Classification in Eight Languages](https://mlsanity.com/paper/1805.09821/) (2018)
- [Universal Sentence Representation Learning with Conditional Masked Language Model](https://mlsanity.com/paper/2012.14388/) (2020)
- [Pairwise Supervised Contrastive Learning of Sentence Representations](https://mlsanity.com/paper/2109.05424/) (2021)
- [mLUKE: The Power of Entity Representations in Multilingual Pretrained Language Models](https://mlsanity.com/paper/2110.08151/) (2021)
- [XLM-K: Improving Cross-Lingual Language Model Pre-training with Multilingual Knowledge](https://mlsanity.com/paper/2109.12573/) (2021)

## Built on by (most cited first, 21 of 21)

- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [When Large Language Models Meet Vector Databases: A Survey](https://mlsanity.com/paper/2402.01763/) (2024)
- [Grounding Language Model with Chunking-Free In-Context Retrieval](https://mlsanity.com/paper/2402.09760/) (2024)
- [Mix-of-Granularity: Optimize the Chunking Granularity for Retrieval-Augmented Generation](https://mlsanity.com/paper/2406.00456/) (2024)
- [Composition-contrastive Learning for Sentence Embeddings](https://mlsanity.com/paper/2307.07380/) (2023)
- [A Comprehensive Survey of Sentence Representations: From the BERT Epoch to the ChatGPT Era and Beyond](https://mlsanity.com/paper/2305.12641/) (2023)
- [Efficient Domain Adaptation of Sentence Embeddings Using Adapters](https://mlsanity.com/paper/2307.03104/) (2023)
- [Assessing News Thumbnail Representativeness: Counterfactual text can enhance the cross-modal matching ability](https://mlsanity.com/paper/2402.11159/) (2024)
- [On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Survey](https://mlsanity.com/paper/2507.20783/) (2025)
- [Dial2vec: Self-Guided Contrastive Learning of Unsupervised Dialogue Embeddings](https://mlsanity.com/paper/2210.15332/) (2022)
- [MIPIC: Matryoshka Representation Learning via Self-Distilled Intra-Relational and Progressive Information Chaining](https://mlsanity.com/paper/2604.24374/) (2026)
- [Analysis of Utterance Embeddings and Clustering Methods Related to Intent Induction for Task-Oriented Dialogue](https://mlsanity.com/paper/2212.02021/) (2022)
- [ESCL: Equivariant Self-Contrastive Learning for Sentence Representations](https://mlsanity.com/paper/2303.05143/) (2023)
- [Adapting Sentence Transformers for the Aviation Domain](https://mlsanity.com/paper/2305.09556/) (2023)
- [Template-assisted Contrastive Learning of Task-oriented Dialogue Sentence Embeddings](https://mlsanity.com/paper/2305.14299/) (2023)
- [SKETCH: Structured Knowledge Enhanced Text Comprehension for Holistic Retrieval](https://mlsanity.com/paper/2412.15443/) (2024)
- [MIC: Maximizing Informational Capacity in Adaptive Representations via Isotropic Subspace Alignment](https://mlsanity.com/paper/2605.29987/) (2026)
- [MED-SE: Medical Entity Definition-based Sentence Embedding](https://mlsanity.com/paper/2212.04734/) (2022)
- [KEPLET: Knowledge-Enhanced Pretrained Language Model with Topic Entity Awareness](https://mlsanity.com/paper/2305.01810/) (2023)
- [Semi-supervised News Discourse Profiling with Contrastive Learning](https://mlsanity.com/paper/2309.11692/) (2023)
- [JFinTEB: Japanese Financial Text Embedding Benchmark](https://mlsanity.com/paper/2604.15882/) (2026)

## Abstract (arXiv)

We present EASE, a novel method for learning sentence embeddings via contrastive learning between sentences and their related entities. The advantage of using entity supervision is twofold: (1) entities have been shown to be a strong indicator of text semantics and thus should provide rich training signals for sentence embeddings; (2) entities are defined independently of languages and thus offer useful cross-lingual alignment supervision. We evaluate EASE against other unsupervised models both in monolingual and multilingual settings. We show that EASE exhibits competitive or better performance in English semantic textual similarity (STS) and short text clustering (STC) tasks and it significantly outperforms baseline methods in multilingual settings on a variety of tasks. Our source code, pre-trained models, and newly constructed multilingual STC dataset are available at https://github.com/studio-ousia/ease.