mlsanityData to 7 Oct 2026

Learning Distributed Representations of Sentences from Unlabelled Data

arXiv 1602.03483 ↗First posted 10 Feb 2016cs.CL cs.LGcited by 90

Not on the map. Why?

Abstract (arXiv)

Unsupervised methods for learning distributed representations of words are ubiquitous in today's NLP research, but far less is known about the best ways to learn distributed phrase or sentence representations from unlabelled data. This paper is a systematic comparison of models that learn such representations. We find that the optimal approach depends critically on the intended application. Deeper, more complex models are preferable for representations to be used in supervised systems, but shallow log-linear models work best for building representation spaces that can be decoded with simple spatial distance metrics. We also propose two new unsupervised representation-learning objectives designed to optimise the trade-off between training time, domain portability and performance.

Key facts. Cited by 90 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Feb 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 23; 2022: 22; 2023: 26; 2024: 11; 2025: 7; 2026: 1.

90 papers.

Sort:

By area:

  1. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  2. 2021ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfercited by 231
  3. 2022DiffCSE: Difference-based Contrastive Learning for Sentence Embeddingscited by 131
  4. 2021TSDAE: Using Transformer-based Sequential Denoising Auto-Encoder for Unsupervised Sentence Embedding Learningcited by 116
  5. 2023AnglE-optimized Text Embeddingscited by 98
  6. 2021Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoderscited by 88
  7. 2023Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentencecited by 81
  8. 2022Meaning without reference in large language modelscited by 76
  9. 2021ESimCSE: Enhanced Sample Building Method for Contrastive Learning of Unsupervised Sentence Embeddingcited by 73
  10. 2022RankGen: Improving Text Generation with Large Ranking Modelscited by 72
  11. 2022Debiased Contrastive Learning of Unsupervised Sentence Representationscited by 56
  12. 2024GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embedding Fine-tuningcited by 40
38 more
  1. 2021Evaluation of BERT and ALBERT Sentence Embedding Performance on Downstream NLP Taskscited by 32
  2. 2022Improved Universal Sentence Embeddings with Prompt-based Contrastive Learning and Energy-based Learningcited by 30
  3. 2022PCL: Peer-Contrastive Learning with Diverse Augmentations for Unsupervised Sentence Embeddingscited by 29
  4. 2022InfoCSE: Information-aggregated Contrastive Learning of Sentence Embeddingscited by 25
  5. 2023RankCSE: Unsupervised Sentence Representations Learning via Learning to Rankcited by 23
  6. 2021Virtual Augmentation Supported Contrastive Learning of Sentence Representationscited by 21
  7. 2023Whitening-based Contrastive Learning of Sentence Embeddingscited by 21
  8. 2021Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillationscited by 20
  9. 2022Just Rank: Rethinking Evaluation with Word and Sentence Similaritiescited by 20
  10. 2021Paraphrastic Representations at Scalecited by 19
  11. 2021DialogueCSE: Dialogue-based Contrastive Learning of Sentence Embeddingscited by 19
  12. 2022Generate, Discriminate and Contrast: A Semi-Supervised Sentence Representation Learning Frameworkcited by 18
  13. 2023Alleviating Over-smoothing for Unsupervised Sentence Representationcited by 16
  14. 2022Ranking-Enhanced Unsupervised Sentence Representation Learningcited by 15
  15. 2023SimCSE++: Improving Contrastive Learning for Sentence Embeddings from Two Perspectivescited by 15
  16. 2021Syntactic Perturbations Reveal Representational Correlates of Hierarchical Phrase Structure in Pretrained Language Modelscited by 14
  17. 2023Composition-contrastive Learning for Sentence Embeddingscited by 14
  18. 2024KDMCSE: Knowledge Distillation Multimodal Sentence Embeddings with Adaptive Angular margin Contrastive Learningcited by 13
  19. 2021Sentence Similarity Based on Contextscited by 12
  20. 2023Language Models are Universal Embedderscited by 11
  21. 2021Word-level Human Interpretable Scoring Mechanism for Novel Text Detection Using Tsetlin Machinescited by 10
  22. 2021Transferring Knowledge from Text to Video: Zero-Shot Anticipation for Procedural Actionscited by 10
  23. 2021Shaking Syntactic Trees on the Sesame Street: Multilingual Probing with Controllable Perturbationscited by 10
  24. 2021Eth2Vec: Learning Contract-Wide Code Representations for Vulnerability Detection on Ethereum Smart Contractscited by 7
  25. 2023CrisisTransformers: Pre-trained language models and sentence encoders for crisis-related social media textscited by 7
  26. 2021Natural Language Understanding for Argumentative Dialogue Systems in the Opinion Building Domaincited by 6
  27. 2022NMTScore: A Multilingual Analysis of Translation-based Text Similarity Measurescited by 6
  28. 2022Contrastive Learning with Prompt-derived Virtual Semantic Prototypes for Unsupervised Sentence Embeddingcited by 6
  29. 2023Efficient Domain Adaptation of Sentence Embeddings Using Adapterscited by 6
  30. 2021Unsupervised Document Embedding via Contrastive Augmentationcited by 5
  31. 2021Semantic sentence similarity: size does not always mattercited by 5
  32. 2021PAUSE: Positive and Annealed Unlabeled Sentence Embeddingcited by 5
  33. 2022Is Encoder-Decoder Redundant for Neural Machine Translation?cited by 5
  34. 2023Clustering-Aware Negative Sampling for Unsupervised Sentence Representationcited by 5
  35. 2023Bridging Continuous and Discrete Spaces: Interpretable Sentence Representation Learning via Compositional Operationscited by 5
  36. 2023AspectCSE: Sentence Embeddings for Aspect-based Semantic Textual Similarity Using Contrastive Learning and Structured Knowledgecited by 5
  37. 2022Retrofitting Multilingual Sentence Embeddings with Abstract Meaning Representationcited by 4
  38. 2023The Daunting Dilemma with Sentence Encoders: Success on Standard Benchmarks, Failure in Capturing Basic Semantic Propertiescited by 4

50 most cited of 90 All 90 (JSON)