mlsanityData to 14 Oct 2026

Improving Contrastive Learning of Sentence Embeddings from AI Feedback

arXiv 2305.01918 ↗First posted 3 May 2023cs.CL cs.AIcited by 29

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Contrastive learning has become a popular approach in natural language processing, particularly for the learning of sentence embeddings. However, the discrete nature of natural language makes it difficult to ensure the quality of positive and negative sample pairs generated through data augmentation methods. Although supervised contrastive learning can produce more accurate sample pairs with human feedback labels, it still lacks fine-grained training signals. In this paper, we propose to improve Contrastive Learning of sentence embeddings from AI Feedback (CLAIF). Our method utilizes AI feedback from large pre-trained language models (LLMs) to construct sample pairs with fine-grained sample similarity scores to improve contrastive learning. Besides, we combine human feedback and AI feedback to provide better supervision signals for supervised contrastive learning of sentence embeddings. Experimental results show that our method achieves state-of-the-art performance on several semantic textual similarity (STS) and transfer learning tasks compared to other unsupervised and supervised contrastive learning methods.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 32 of its 57 references that are in the map. First posted on arXiv 3 May 2023.

Builds on

32 of 57 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Training language models to follow instructions with human feedback4 Mar

Other references · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  7. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  8. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  9. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  10. 2020Supervised Contrastive Learningcited by 2,829
  11. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  12. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
19 more
  1. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  2. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 1,001
  3. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 505
  4. 2018SentEval: An Evaluation Toolkit for Universal Sentence Representationscited by 294
  5. 2021ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generationcited by 260
  6. 2019Representation Degeneration Problem in Training Natural Language Generation Modelscited by 255
  7. 2021ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfercited by 231
  8. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 211
  9. 2021Whitening Sentence Representations for Better Semantics and Faster Retrievalcited by 202
  10. 2022Generating Training Data with Language Models: Towards Zero-Shot Language Understandingcited by 199
  11. 2021Generating Datasets with Pretrained Language Modelscited by 193
  12. 2020Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Taskscited by 138
  13. 2022DiffCSE: Difference-based Contrastive Learning for Sentence Embeddingscited by 132
  14. 2022PromptBERT: Improving BERT Sentence Embeddings with Promptscited by 75
  15. 2021ESimCSE: Enhanced Sample Building Method for Contrastive Learning of Unsupervised Sentence Embeddingcited by 74
  16. 2022Debiased Contrastive Learning of Unsupervised Sentence Representationscited by 56
  17. 2022ERNIE-Search: Bridging Cross-Encoder with Dual-Encoder via Self On-the-fly Distillation for Dense Passage Retrievalcited by 41
  18. 2022SBERT studies Meaning Representations: Decomposing Sentence Embeddings into Explainable Semantic Featurescited by 33
  19. 2021Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillationscited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 12; 2024: 9; 2025: 6; 2026: 2.

29 papers.

Sort:

By area:

  1. 2023Unified Demonstration Retriever for In-Context Learningcited by 142
  2. 2023Scaling Sentence Embeddings with Large Language Modelscited by 85
  3. 2023ClusterLLM: Large Language Models as a Guide for Text Clusteringcited by 70
  4. 2024GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embedding Fine-tuningcited by 40
  5. 2023Large Language Models as Annotators: Enhancing Generalization of NLP Models at Minimal Costcited by 33
  6. 2023Evaluating Hallucinations in Chinese Large Language Modelscited by 33
  7. 2024HalluDial: A Large-Scale Benchmark for Automatic Dialogue-Level Hallucination Evaluationcited by 27
  8. 2024LLMvsSmall Model? Large Language Model Based Text Augmentation Enhanced Personality Detection Modelcited by 21
  9. 2024Meta-Task Prompting Elicits Embeddings from Large Language Modelscited by 16
  10. 2023Generalized Category Discovery with Large Language Models in the Loopcited by 15
  11. 2025Diffusion Models Through a Global Lens: Are They Culturally Inclusive?cited by 14
  12. 2023Language Models are Universal Embedderscited by 11
17 more
  1. 2023Controllable Data Augmentation for Few-Shot Text Mining with Chain-of-Thought Attribute Manipulationcited by 9
  2. 2025GLEAN: Active Generalized Category Discovery with Diverse LLM Feedbackcited by 6
  3. 2026CSRv2: Unlocking Ultra-Sparse Embeddingscited by 6
  4. 2023ZYN: Zero-Shot Reward Models with Yes-No Questions for RLAIFcited by 4
  5. 2024DenoSent: A Denoising Objective for Self-Supervised Sentence Representation Learningcited by 4
  6. 2024Large Language Model Augmented Exercise Retrieval for Personalized Language Learningcited by 4
  7. 2024Enhancing Unsupervised Sentence Embeddings via Knowledge-Driven Data Augmentation and Gaussian-Decayed Contrastive Learningcited by 4
  8. 2023Large Language Models can Contrastively Refine their Generation for Better Sentence Representation Learningcited by 3
  9. 2024Universal New Physics Latent Spacecited by 3
  10. 2025On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Surveycited by 3
  11. 2024Exploring ChatGPT-based Augmentation Strategies for Contrastive Aspect-based Sentiment Analysiscited by 2
  12. 2025DALR: Dual-level Alignment Learning for Multimodal Sentence Representation Learningcited by 2
  13. 2025BiXSE: Improving Dense Retrieval via Probabilistic Graded Relevance Distillationcited by 2
  14. 2023Template-assisted Contrastive Learning of Task-oriented Dialogue Sentence Embeddingscited by 1
  15. 2023Narrowing the Gap between Supervised and Unsupervised Sentence Representation Learning with Large Language Modelcited by 0
  16. 2025Navigation-GPT: A Robust and Adaptive Framework Utilizing Large Language Models for Navigation Applicationscited by 0
  17. 2026ttda704 at SemEval-2026 Task 4: Modeling Narrative Structures via Pseudonymization and Multi-View Sentence Alignmentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021DistilCSE: Effective Knowledge Distillation For Contrastive Sentence Embeddings
  2. 2024KDMCSE: Knowledge Distillation Multimodal Sentence Embeddings with Adaptive Angular margin Contrastive Learning
  3. 2025Let LLMs Speak Embedding Languages: Generative Text Embeddings via Iterative Contrastive Refinement
  4. 2024Advancing Semantic Textual Similarity Modeling: A Regression Framework with Translated ReLU and Smooth K2 Loss
  5. 2025Following the Autoregressive Nature of LLM Embeddings via Compression and Alignment
  6. 2024Contrastive Learning in Distilled Models
  7. 2023DistillCSE: Distilled Contrastive Learning for Sentence Embeddings
  8. 2023RankCSE: Unsupervised Sentence Representations Learning via Learning to Rank