mlsanityData to 7 Oct 2026

Ranking-Enhanced Unsupervised Sentence Representation Learning

arXiv 2209.04333 ↗First posted 9 Sep 2022cs.CLcited by 15

Not on the map. Why?

Abstract (arXiv)

Unsupervised sentence representation learning has progressed through contrastive learning and data augmentation methods such as dropout masking. Despite this progress, sentence encoders are still limited to using only an input sentence when predicting its semantic vector. In this work, we show that the semantic meaning of a sentence is also determined by nearest-neighbor sentences that are similar to the input sentence. Based on this finding, we propose a novel unsupervised sentence encoder, RankEncoder. RankEncoder predicts the semantic vector of an input sentence by leveraging its relationship with other sentences in an external corpus, as well as the input sentence itself. We evaluate RankEncoder on semantic textual benchmark datasets. From the experimental results, we verify that 1) RankEncoder achieves 80.07% Spearman's correlation, a 1.1% absolute improvement compared to the previous state-of-the-art performance, 2) RankEncoder is universally applicable to existing unsupervised sentence embedding methods, and 3) RankEncoder is specifically effective for predicting the similarity scores of similar sentence pairs.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 26 of its 42 references that are in the map. First posted on arXiv 9 Sep 2022.

Builds on

26 of 42 references.

References · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  7. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
  8. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  9. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  10. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  11. 2018Universal Sentence Encodercited by 485
  12. 2017Supervised Learning of Universal Sentence Representations from Natural Language Inference Datacited by 428
14 more
  1. 2015Skip-Thought Vectorscited by 321
  2. 2018SentEval: An Evaluation Toolkit for Universal Sentence Representationscited by 294
  3. 2021ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfercited by 231
  4. 2018An efficient framework for learning sentence representationscited by 197
  5. 2020Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Taskscited by 136
  6. 2022DiffCSE: Difference-based Contrastive Learning for Sentence Embeddingscited by 131
  7. 2021Self-Guided Contrastive Learning for BERT Sentence Representationscited by 103
  8. 2016Learning Distributed Representations of Sentences from Unlabelled Datacited by 90
  9. 2021Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoderscited by 88
  10. 2022PromptBERT: Improving BERT Sentence Embeddings with Promptscited by 75
  11. 2021ESimCSE: Enhanced Sample Building Method for Contrastive Learning of Unsupervised Sentence Embeddingcited by 73
  12. 2022Debiased Contrastive Learning of Unsupervised Sentence Representationscited by 56
  13. 2022SNCSE: Contrastive Learning for Unsupervised Sentence Embedding with Soft Negative Samplescited by 39
  14. 2021Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillationscited by 20

Built on by

Cited by year

Citing papers per year: 2023: 8; 2024: 3; 2025: 4.

15 papers.

Sort:

By area:

  1. 2023Text Classification via Large Language Modelscited by 104
  2. 2024RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language Modelscited by 84
  3. 2023Pushing the Limits of ChatGPT on NLP Taskscited by 27
  4. 2023A Comprehensive Survey of Sentence Representations: From the BERT Epoch to the ChatGPT Era and Beyondcited by 11
  5. 2023CoT-BERT: Enhancing Unsupervised Sentence Representation through Chain-of-Thoughtcited by 8
  6. 2023Revealing the Blind Spot of Sentence Encoder Evaluation by HEROScited by 5
  7. 2024AAdaM at SemEval-2024 Task 1: Augmentation and Adaptation for Multilingual Semantic Textual Relatednesscited by 4
  8. 2023Large Language Models can Contrastively Refine their Generation for Better Sentence Representation Learningcited by 3
  9. 2025On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Surveycited by 3
  10. 2023Unsupervised Sentence Representation Learning with Frequency-induced Adversarial Tuning and Incomplete Sentence Filteringcited by 2
  11. 2025Refining Sentence Embedding Model through Ranking Sentences Generation with Large Language Modelscited by 2
  12. 2025OASIS: Order-Augmented Strategy for Improved Code Searchcited by 2
3 more
  1. 2025DALR: Dual-level Alignment Learning for Multimodal Sentence Representation Learningcited by 2
  2. 2023Generate to Understand for Representationcited by 0
  3. 2024DefSent+: Improving sentence embeddings of language models by projecting definition sentences into a quasi-isotropic or isotropic vector space of unlimited dictionary entriescited by 0