mlsanityData to 14 Oct 2026

Composition-contrastive Learning for Sentence Embeddings

arXiv 2307.07380 ↗First posted 14 Jul 2023cs.CL cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

Vector representations of natural language are ubiquitous in search applications. Recently, various methods based on contrastive learning have been proposed to learn textual representations from unlabelled data; by maximizing alignment between minimally-perturbed embeddings of the same text, and encouraging a uniform distribution of embeddings across a broader corpus. Differently, we propose maximizing alignment between texts and a composition of their phrasal constituents. We consider several realizations of this objective and elaborate the impact on representations in each case. Experimental results on semantic textual similarity tasks show improvements over baselines that are comparable with state-of-the-art approaches. Moreover, this work is the first to do so without incurring costs in auxiliary training objectives or additional network parameters.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 38 of its 51 references that are in the map. First posted on arXiv 14 Jul 2023.

Builds on

38 of 51 references.

References · 38

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  8. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  9. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  10. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 1,001
  11. 2020What Makes for Good Views for Contrastive Learning?cited by 846
  12. 2020Contrastive Learning with Hard Negative Samplescited by 628
26 more
  1. 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
  2. 2017Supervised Learning of Universal Sentence Representations from Natural Language Inference Datacited by 432
  3. 2020Hard Negative Mixing for Contrastive Learningcited by 402
  4. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 378
  5. 2015Skip-Thought Vectorscited by 322
  6. 2018SentEval: An Evaluation Toolkit for Universal Sentence Representationscited by 294
  7. 2020DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representationscited by 251
  8. 2021ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfercited by 231
  9. 2018An efficient framework for learning sentence representationscited by 198
  10. 2020Predicting What You Already Know Helps: Provable Self-Supervised Learningcited by 152
  11. 2020Contrastive learning, multi-view redundancy, and linear modelscited by 146
  12. 2020Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biasescited by 143
  13. 2022DiffCSE: Difference-based Contrastive Learning for Sentence Embeddingscited by 132
  14. 2021Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learningcited by 119
  15. 2021Can contrastive learning avoid shortcut solutions?cited by 114
  16. 2016Learning Distributed Representations of Sentences from Unlabelled Datacited by 91
  17. 2021ESimCSE: Enhanced Sample Building Method for Contrastive Learning of Unsupervised Sentence Embeddingcited by 74
  18. 2022Debiased Contrastive Learning of Unsupervised Sentence Representationscited by 56
  19. 2020On Mutual Information in Contrastive Learning for Visual Representationscited by 54
  20. 2021Understanding Hard Negatives in Noise Contrastive Estimationcited by 39
  21. 2022SNCSE: Contrastive Learning for Unsupervised Sentence Embedding with Soft Negative Samplescited by 39
  22. 2022Improved Universal Sentence Embeddings with Prompt-based Contrastive Learning and Energy-based Learningcited by 30
  23. 2022PCL: Peer-Contrastive Learning with Diverse Augmentations for Unsupervised Sentence Embeddingscited by 29
  24. 2022InfoCSE: Information-aggregated Contrastive Learning of Sentence Embeddingscited by 26
  25. 2022Exploring the Impact of Negative Samples of Contrastive Learning: A Case Study of Sentence Embeddingcited by 25
  26. 2022EASE: Entity-Aware Contrastive Learning of Sentence Embeddingcited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 5; 2025: 5; 2026: 1.

14 papers.

Sort:

By area:

  1. 2023A Semantic Invariant Robust Watermark for Large Language Modelscited by 112
  2. 2023AnglE-optimized Text Embeddingscited by 99
  3. 2024Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Modelscited by 34
  4. 2024Can Watermarks Survive Translation? On the Cross-lingual Consistency of Text Watermark for Large Language Modelscited by 33
  5. 2024Token Prepending: A Training-Free Approach for Eliciting Better Sentence Embeddings from LLMscited by 18
  6. 2025Contrastive Prompting Enhances Sentence Embeddings in LLMs through Inference-Time Steeringcited by 13
  7. 2024Adversarial Style Augmentation via Large Language Model for Robust Fake News Detectioncited by 6
  8. 2025PRO: Enabling Precise and Robust Text Watermark for Open-Source LLMscited by 4
  9. 2025Hierarchical Token Prepending: Enhancing Information Flow in Decoder-based LLM Embeddingscited by 3
  10. 2024Enhancing Contrastive Learning Inspired by the Philosophy of "The Blind Men and the Elephant"cited by 1
  11. 2023Semi-supervised News Discourse Profiling with Contrastive Learningcited by 0
  12. 2025DualGuard: Dual-stream Large Language Model Watermarking Defense against Paraphrase and Spoofing Attackcited by 0
2 more
  1. 2025ARC: Leveraging Compositional Representations for Cross-Problem Learning on VRPscited by 0
  2. 2026CWoMP: Morpheme Representation Learning for Interlinear Glossingcited by 0