mlsanityData to 14 Oct 2026

Skip-Thought Vectors

arXiv 1506.06726 ↗First posted 22 Jun 2015cs.CL cs.LGcited by 322

Not on the map. Why?

Abstract (arXiv)

We describe an approach for unsupervised learning of a generic, distributed sentence encoder. Using the continuity of text from books, we train an encoder-decoder model that tries to reconstruct the surrounding sentences of an encoded passage. Sentences that share semantic and syntactic properties are thus mapped to similar vector representations. We next introduce a simple vocabulary expansion method to encode words that were not seen as part of training, allowing us to expand our vocabulary to a million words. After training our model, we extract and evaluate our vectors with linear models on 8 tasks: semantic relatedness, paraphrase detection, image-sentence ranking, question-type classification and 4 benchmark sentiment and subjectivity datasets. The end result is an off-the-shelf encoder that can produce highly generic sentence representations that are robust and perform well in practice. We will make our encoder publicly available.

Key facts. Cited by 322 papers in the map (data through 14 Oct 2026). First posted on arXiv 22 Jun 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 1; 2020: 1; 2021: 98; 2022: 78; 2023: 73; 2024: 41; 2025: 23; 2026: 7.

322 papers.

Sort:

By area:

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  3. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  4. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  5. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
  6. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 463
  7. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 361
  8. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 333
  9. 2021ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfercited by 231
  10. 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Taskscited by 228
  11. 2021Generating Datasets with Pretrained Language Modelscited by 193
  12. 2023Text Embeddings Reveal (Almost) As Much As Textcited by 192
38 more
  1. 2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
  2. 2021The Low-Rank Simplicity Bias in Deep Networkscited by 154
  3. 2023Large-scale Multi-Modal Pre-trained Models: A Comprehensive Surveycited by 153
  4. 2022Scaling Laws and Interpretability of Learning from Repeated Datacited by 152
  5. 2022DiffCSE: Difference-based Contrastive Learning for Sentence Embeddingscited by 132
  6. 2023DDCoT: Duty-Distinct Chain-of-Thought Prompting for Multimodal Reasoning in Language Modelscited by 128
  7. 2022Does CLIP Bind Concepts? Probing Compositionality in Large Image Modelscited by 102
  8. 2023AnglE-optimized Text Embeddingscited by 99
  9. 2021A Comprehensive Assessment of Dialog Evaluation Metricscited by 90
  10. 2021Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoderscited by 88
  11. 2023Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentencecited by 81
  12. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 76
  13. 2021FairFil: Contrastive Neural Debiasing Method for Pretrained Text Encoderscited by 75
  14. 2021ESimCSE: Enhanced Sample Building Method for Contrastive Learning of Unsupervised Sentence Embeddingcited by 74
  15. 2022RankGen: Improving Text Generation with Large Ranking Modelscited by 72
  16. 2024Repetition Improves Language Model Embeddingscited by 72
  17. 2021Generating Smooth Pose Sequences for Diverse Human Motion Predictioncited by 66
  18. 2023Lifelong Language Pretraining with Distribution-Specialized Expertscited by 66
  19. 2021SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognitioncited by 60
  20. 2022Twin Contrastive Learning for Online Clusteringcited by 60
  21. 2021BlonDe: An Automatic Evaluation Metric for Document-level Machine Translationcited by 58
  22. 2022Debiased Contrastive Learning of Unsupervised Sentence Representationscited by 56
  23. 2021Adversarial Text-to-Image Synthesis: A Reviewcited by 52
  24. 2023Gloss Attention for Gloss-free Sign Language Translationcited by 51
  25. 2021PalmTree: Learning an Assembly Language Model for Instruction Embeddingcited by 49
  26. 2021Understanding Negative Samples in Instance Discriminative Self-supervised Representation Learningcited by 47
  27. 2021Long Text Generation by Modeling Sentence-Level and Discourse-Level Coherencecited by 45
  28. 2022Datasheet for the Pilecited by 43
  29. 2023Towards Surveillance Video-and-Language Understanding: New Dataset, Baselines, and Challengescited by 42
  30. 2024KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharingcited by 41
  31. 2024GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embedding Fine-tuningcited by 40
  32. 2023Transformers in Speech Processing: A Surveycited by 39
  33. 2022Temporal Sentence Grounding in Videos: A Survey and Future Directionscited by 37
  34. 2024Your Mixture-of-Experts LLM Is Secretly an Embedding Model For Freecited by 37
  35. 2021Relation-aware Instance Refinement for Weakly Supervised Visual Groundingcited by 35
  36. 2021Multi-level Motion Attention for Human Motion Predictioncited by 35
  37. 2023Learning Visual Representations via Language-Guided Samplingcited by 32
  38. 2024LLMs are Also Effective Embedding Models: An In-depth Overviewcited by 32

50 most cited of 322 All 322 (JSON)