mlsanityData to 7 Oct 2026

Supervised Learning of Universal Sentence Representations from Natural Language Inference Data

arXiv 1705.02364 ↗First posted 5 May 2017cs.CLcited by 428

Not on the map. Why?

Abstract (arXiv)

Many modern NLP systems rely on word embeddings, previously trained in an unsupervised manner on large corpora, as base features. Efforts to obtain embeddings for larger chunks of text, such as sentences, have however not been so successful. Several attempts at learning unsupervised representations of sentences have not reached satisfactory enough performance to be widely adopted. In this paper, we show how universal sentence representations trained using the supervised data of the Stanford Natural Language Inference datasets can consistently outperform unsupervised methods like SkipThought vectors on a wide range of transfer tasks. Much like how computer vision uses ImageNet to obtain features, which can then be transferred to other tasks, our work tends to indicate the suitability of natural language inference for transfer learning to other NLP tasks. Our encoder is publicly available.

Key facts. Cited by 428 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 2; 2021: 119; 2022: 103; 2023: 95; 2024: 55; 2025: 30; 2026: 23.

428 papers.

Sort:

By area:

  1. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  2. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  3. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 705
  4. 2022MTEB: Massive Text Embedding Benchmarkcited by 623
  5. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 462
  6. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 461
  7. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 396
  8. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 329
  9. 2023Improving Text Embeddings with Large Language Modelscited by 295
  10. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 283
  11. 2022GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Recordscited by 238
  12. 2021ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfercited by 231
38 more
  1. 2024Generative Representational Instruction Tuningcited by 227
  2. 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Taskscited by 222
  3. 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 217
  4. 2021Whitening Sentence Representations for Better Semantics and Faster Retrievalcited by 198
  5. 2021Generating Datasets with Pretrained Language Modelscited by 192
  6. 2023MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrievalcited by 182
  7. 2021Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answeringcited by 178
  8. 2021Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Docited by 178
  9. 2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
  10. 2022DiffCSE: Difference-based Contrastive Learning for Sentence Embeddingscited by 131
  11. 2022SciRepEval: A Multi-Format Benchmark for Scientific Document Representationscited by 124
  12. 2021TSDAE: Using Transformer-based Sequential Denoising Auto-Encoder for Unsupervised Sentence Embedding Learningcited by 116
  13. 2021KLUE: Korean Language Understanding Evaluationcited by 113
  14. 2023SONAR: Sentence-Level Multimodal and Language-Agnostic Representationscited by 105
  15. 2021Self-Guided Contrastive Learning for BERT Sentence Representationscited by 103
  16. 2022Does CLIP Bind Concepts? Probing Compositionality in Large Image Modelscited by 101
  17. 2023AnglE-optimized Text Embeddingscited by 98
  18. 2020Human-centric Dialog Training via Offline Reinforcement Learningcited by 92
  19. 2021Evaluating the Robustness of Neural Language Models to Input Perturbationscited by 87
  20. 2022Same Author or Just Same Topic? Towards Content-Independent Style Representationscited by 83
  21. 2023Scaling Sentence Embeddings with Large Language Modelscited by 83
  22. 2023Provable Dynamic Fusion for Low-Quality Multimodal Datacited by 81
  23. 2022An Information-Theoretic Approach to Transferability in Task Transfer Learningcited by 79
  24. 2021Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Modelscited by 78
  25. 2021FairFil: Contrastive Neural Debiasing Method for Pretrained Text Encoderscited by 75
  26. 2022PromptBERT: Improving BERT Sentence Embeddings with Promptscited by 75
  27. 2021CoSQA: 20,000+ Web Queries for Code Search and Question Answeringcited by 74
  28. 2024GuardT2I: Defending Text-to-Image Models from Adversarial Promptscited by 61
  29. 2022Debiased Contrastive Learning of Unsupervised Sentence Representationscited by 56
  30. 2021Automated Fact-Checking: A Surveycited by 55
  31. 2023Gloss Attention for Gloss-free Sign Language Translationcited by 50
  32. 2021CombOptNet: Fit the Right NP-Hard Problem by Learning Integer Programming Constraintscited by 49
  33. 2021Knowing More About Questions Can Help: Improving Calibration in Question Answeringcited by 47
  34. 2022Temporal Sentence Grounding in Videos: A Survey and Future Directionscited by 37
  35. 2021STraTA: Self-Training with Task Augmentation for Better Few-shot Learningcited by 33
  36. 2021Evaluation of BERT and ALBERT Sentence Embedding Performance on Downstream NLP Taskscited by 32
  37. 2023BeLLM: Backward Dependency Enhanced Large Language Model for Sentence Embeddingscited by 32
  38. 2021Survey on reinforcement learning for language processingcited by 31

50 most cited of 428 All 428 (JSON)