mlsanityData to 7 Oct 2026

An efficient framework for learning sentence representations

arXiv 1803.02893 ↗First posted 7 Mar 2018cs.CL cs.AI cs.LGcited by 197

Not on the map. Why?

Abstract (arXiv)

In this work we propose a simple and efficient framework for learning sentence representations from unlabelled data. Drawing inspiration from the distributional hypothesis and recent work on learning sentence representations, we reformulate the problem of predicting the context in which a sentence appears as a classification problem. Given a sentence and its context, a classifier distinguishes context sentences from other contrastive sentences based on their vector representations. This allows us to efficiently learn different types of encoding functions, and we show that the model learns high-quality sentence representations. We demonstrate that our sentence representations outperform state-of-the-art unsupervised and supervised representation learning methods on several downstream NLP tasks that involve understanding sentence semantics while achieving an order of magnitude speedup in training time.

Key facts. Cited by 197 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 58; 2022: 57; 2023: 37; 2024: 25; 2025: 16; 2026: 4.

197 papers.

Sort:

By area:

  1. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  2. 2021TS2Vec: Towards Universal Representation of Time Seriescited by 463
  3. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 462
  4. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
  5. 2022Making the Most of Text Semantics to Improve Biomedical Vision--Language Processingcited by 336
  6. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 329
  7. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 308
  8. 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Taskscited by 222
  9. 2021Contrastive Learning Inverts the Data Generating Processcited by 220
  10. 2022DiffCSE: Difference-based Contrastive Learning for Sentence Embeddingscited by 131
  11. 2021Proof Artifact Co-training for Theorem Proving with Language Modelscited by 130
  12. 2021Self-Guided Contrastive Learning for BERT Sentence Representationscited by 103
38 more
  1. 2022Understanding Contrastive Learning Requires Incorporating Inductive Biasescited by 96
  2. 2021Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoderscited by 88
  3. 2021CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIPcited by 88
  4. 2021Divide and Contrast: Self-supervised Learning from Uncurated Datacited by 85
  5. 2021ESimCSE: Enhanced Sample Building Method for Contrastive Learning of Unsupervised Sentence Embeddingcited by 73
  6. 2021AutoGCL: Automated Graph Contrastive Learning via Learnable View Generatorscited by 73
  7. 2022RankGen: Improving Text Generation with Large Ranking Modelscited by 72
  8. 2022CLIPPO: Image-and-Language Understanding from Pixels Onlycited by 53
  9. 2021CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understandingcited by 50
  10. 2021Contrastive Learning for Neural Topic Modelcited by 48
  11. 2024Optimized Feature Generation for Tabular Data via LLMs with Decision Tree Reasoningcited by 48
  12. 2021Understanding Negative Samples in Instance Discriminative Self-supervised Representation Learningcited by 47
  13. 2022Robust Contrastive Learning against Noisy Viewscited by 45
  14. 2021A Primer on Contrastive Pretraining in Language Processing: Methods, Lessons Learned and Perspectivescited by 41
  15. 2023Contrastive Difference Predictive Codingcited by 35
  16. 2023Learning Visual Representations via Language-Guided Samplingcited by 32
  17. 2024Towards Few-Shot Adaptation of Foundation Models via Multitask Finetuningcited by 31
  18. 2022Improved Universal Sentence Embeddings with Prompt-based Contrastive Learning and Energy-based Learningcited by 30
  19. 2022PCL: Peer-Contrastive Learning with Diverse Augmentations for Unsupervised Sentence Embeddingscited by 29
  20. 2022Geodesic Multi-Modal Mixup for Robust Fine-Tuningcited by 29
  21. 2021Max-Margin Contrastive Learningcited by 27
  22. 2023Scene Graph as Pivoting: Inference-time Image-free Unsupervised Multimodal Machine Translation with Visual Scene Hallucinationcited by 27
  23. 2024Heterogeneous Contrastive Learning for Foundation Models and Beyondcited by 27
  24. 2022English Contrastive Learning Can Learn Universal Cross-lingual Sentence Embeddingscited by 26
  25. 2021On the Surrogate Gap between Contrastive and Supervised Lossescited by 25
  26. 2022InfoCSE: Information-aggregated Contrastive Learning of Sentence Embeddingscited by 25
  27. 2022C2-CRS: Coarse-to-Fine Contrastive Learning for Conversational Recommender Systemcited by 24
  28. 2023InfoCTM: A Mutual Information Maximization Perspective of Cross-Lingual Topic Modelingcited by 24
  29. 2023RankCSE: Unsupervised Sentence Representations Learning via Learning to Rankcited by 23
  30. 2021Simpler, Faster, Stronger: Breaking The log-K Curse On Contrastive Learners With FlatNCEcited by 22
  31. 2021Virtual Augmentation Supported Contrastive Learning of Sentence Representationscited by 21
  32. 2021Embedding Arithmetic of Multimodal Queries for Image Retrievalcited by 21
  33. 2022EASE: Entity-Aware Contrastive Learning of Sentence Embeddingcited by 21
  34. 2023Matrix Information Theory for Self-Supervised Learningcited by 21
  35. 2023Whitening-based Contrastive Learning of Sentence Embeddingscited by 21
  36. 2021Contrastive Mixup: Self- and Semi-Supervised learning for Tabular Domaincited by 20
  37. 2021Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillationscited by 20
  38. 2022Just Rank: Rethinking Evaluation with Word and Sentence Similaritiescited by 20

50 most cited of 197 All 197 (JSON)