mlsanityData to 14 Oct 2026

SCD: Self-Contrastive Decorrelation for Sentence Embeddings

arXiv 2203.07847 ↗First posted 15 Mar 2022cs.CL cs.LGcited by 20

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose Self-Contrastive Decorrelation (SCD), a self-supervised approach. Given an input sentence, it optimizes a joint self-contrastive and decorrelation objective. Learning a representation is facilitated by leveraging the contrast arising from the instantiation of standard dropout at different rates. The proposed method is conceptually simple yet empirically powerful. It achieves comparable results with state-of-the-art methods on multiple benchmarks without using contrastive pairs. This study opens up avenues for efficient self-supervised learning methods that are more robust than current contrastive methods.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 24 of its 41 references that are in the map. First posted on arXiv 15 Mar 2022.

Builds on

24 of 41 references.

References · 24

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  7. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  8. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  9. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  10. 2018Deep contextualized word representationscited by 2,290
  11. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  12. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
12 more
  1. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  2. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  3. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 1,001
  4. 2018SentEval: An Evaluation Toolkit for Universal Sentence Representationscited by 294
  5. 2020Whitening for Self-Supervised Representation Learningcited by 279
  6. 2020DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representationscited by 251
  7. 2021Whitening Sentence Representations for Better Semantics and Faster Retrievalcited by 202
  8. 2020CLEAR: Contrastive Learning for Sentence Representationcited by 186
  9. 2020CERT: Contrastive Self-supervised Learning for Language Understandingcited by 169
  10. 2017Learned in Translation: Contextualized Word Vectorscited by 104
  11. 2020An Unsupervised Sentence Embedding Method by Mutual Information Maximizationcited by 91
  12. 2020Contrastive Self-Supervised Learning for Commonsense Reasoningcited by 25

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 4; 2023: 9; 2024: 4; 2025: 2; 2026: 1.

20 papers.

Sort:

  1. 2023SimCSE++: Improving Contrastive Learning for Sentence Embeddings from Two Perspectivescited by 15
  2. 2023A Comprehensive Survey of Sentence Representations: From the BERT Epoch to the ChatGPT Era and Beyondcited by 12
  3. 2022miCSE: Mutual Information Contrastive Learning for Low-shot Sentence Embeddingscited by 7
  4. 2024Extracting Sentence Embeddings from Pretrained Transformer Modelscited by 7
  5. 2022An Information Minimization Based Contrastive Learning Model for Unsupervised Sentence Embeddings Learningcited by 6
  6. 2023Unsupervised Sentence Representation Learning with Frequency-induced Adversarial Tuning and Incomplete Sentence Filteringcited by 3
  7. 2023Efficient Document Embeddings via Self-Contrastive Bregman Divergence Learningcited by 3
  8. 2025On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Surveycited by 3
  9. 2022A Benchmark Study of Contrastive Learning for Arabic Social Meaningcited by 2
  10. 2022Dial2vec: Self-Guided Contrastive Learning of Unsupervised Dialogue Embeddingscited by 2
  11. 2023Diversified Ensemble of Independent Sub-Networks for Robust Self-Supervised Representation Learningcited by 2
  12. 2023Non-contrastive sentence representations via self-supervisioncited by 2
8 more
  1. 2024Unsupervised hard Negative Augmentation for contrastive learningcited by 2
  2. 2023ESCL: Equivariant Self-Contrastive Learning for Sentence Representationscited by 1
  3. 2023Sentence Representations via Gaussian Embeddingcited by 1
  4. 2023An Efficient Self-Supervised Cross-View Training For Sentence Embeddingcited by 1
  5. 2024UNSEE: Unsupervised Non-contrastive Sentence Embeddingscited by 0
  6. 2024Towards Better Understanding of Contrastive Sentence Representation Learning: A Unified Paradigm for Gradientcited by 0
  7. 2025Skim-Aware Contrastive Learning for Efficient Document Representationcited by 0
  8. 2026Cluster Validation Indices as Self-Supervised Objectives for Text Representation Learningcited by 0