mlsanityData to 14 Oct 2026

Universal Sentence Encoder

arXiv 1803.11175 ↗First posted 29 Mar 2018cs.CLcited by 489

Not on the map. Why?

Abstract (arXiv)

We present models for encoding sentences into embedding vectors that specifically target transfer learning to other NLP tasks. The models are efficient and result in accurate performance on diverse transfer tasks. Two variants of the encoding models allow for trade-offs between accuracy and compute resources. For both variants, we investigate and report the relationship between model complexity, resource consumption, the availability of transfer task training data, and task performance. Comparisons are made with baselines that use word level transfer learning via pretrained word embeddings as well as baselines do not use any transfer learning. We find that transfer learning using sentence embeddings tends to outperform word level transfer. With transfer learning via sentence embeddings, we observe surprisingly good performance with minimal amounts of supervised training data for a transfer task. We obtain encouraging results on Word Embedding Association Tests (WEAT) targeted at detecting model bias. Our pre-trained sentence encoding models are made freely available for download and on TF Hub.

Key facts. Cited by 489 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 108; 2022: 117; 2023: 105; 2024: 79; 2025: 59; 2026: 20.

489 papers.

Sort:

By area:

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  2. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  3. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  4. 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 801
  5. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 463
  6. 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 349
  7. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 234
  8. 2022SGPT: GPT Sentence Embeddings for Semantic Searchcited by 204
  9. 2021Whitening Sentence Representations for Better Semantics and Faster Retrievalcited by 202
  10. 2022Fine-tuning language models to find agreement among humans with diverse preferencescited by 189
  11. 2023MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrievalcited by 185
  12. 2021Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Docited by 178
38 more
  1. 2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
  2. 2023Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functionscited by 131
  3. 2023A Survey of Text Watermarking in the Era of Large Language Modelscited by 112
  4. 2021BERT: A Review of Applications in Natural Language Processing and Understandingcited by 109
  5. 2023SONAR: Sentence-Level Multimodal and Language-Agnostic Representationscited by 108
  6. 2021Self-Guided Contrastive Learning for BERT Sentence Representationscited by 103
  7. 2025A Survey on Vision-Language-Action Models: An Action Tokenization Perspectivecited by 96
  8. 2020Human-centric Dialog Training via Offline Reinforcement Learningcited by 93
  9. 2021Evaluating the Robustness of Neural Language Models to Input Perturbationscited by 87
  10. 2022MuLan: A Joint Embedding of Music Audio and Natural Languagecited by 87
  11. 2024AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agentscited by 87
  12. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 84
  13. 2023Decomposing the Generalization Gap in Imitation Learning for Visual Robotic Manipulationcited by 82
  14. 2021Towards Improving Adversarial Training of NLP Modelscited by 79
  15. 2021A Review on Graph Neural Network Methods in Financial Applicationscited by 78
  16. 2022Neighborhood Contrastive Learning for Scientific Document Representations with Citation Embeddingscited by 75
  17. 2022StoryDALL-E: Adapting Pretrained Text-to-Image Transformers for Story Continuationcited by 74
  18. 2023Adversarial Demonstration Attacks on Large Language Modelscited by 63
  19. 2021The Dawn of Quantum Natural Language Processingcited by 61
  20. 2022Robotic Skill Acquisition via Instruction Augmentation with Vision-Language Modelscited by 58
  21. 2021Searching for an Effective Defender: Benchmarking Defense against Adversarial Word Substitutioncited by 56
  22. 2022BITE: Textual Backdoor Attacks with Iterative Trigger Injectioncited by 53
  23. 2023Towards LLM-guided Causal Explainability for Black-box Text Classifierscited by 44
  24. 2022How Large Language Models are Transforming Machine-Paraphrased Plagiarismcited by 38
  25. 2021ASPECTNEWS: Aspect-Oriented Summarization of News Documentscited by 37
  26. 2022ECCV Caption: Correcting False Negatives by Collecting Machine-and-Human-verified Image-Caption Associations for MS-COCOcited by 37
  27. 2022Deepfake Text Detection: Limitations and Opportunitiescited by 37
  28. 2024Learning to Generate Explainable Stock Predictions using Self-Reflective Large Language Modelscited by 36
  29. 2021Controllable Semantic Parsing via Retrieval Augmentationcited by 35
  30. 2024Claim Detection for Automated Fact-checking: A Survey on Monolingual, Multilingual and Cross-Lingual Researchcited by 35
  31. 2023Text-CRS: A Generalized Certified Robustness Framework against Textual Adversarial Attackscited by 34
  32. 2022Human Evaluation and Correlation with Automatic Metrics in Consultation Note Generationcited by 33
  33. 2021Civil Rephrases Of Toxic Texts With Self-Supervised Transformerscited by 32
  34. 2022Detection of Word Adversarial Examples in Text Classification: Benchmark and Baseline via Robust Density Estimationcited by 32
  35. 2023MathAttack: Attacking Large Language Models Towards Math Solving Abilitycited by 32
  36. 2021Survey on reinforcement learning for language processingcited by 31
  37. 2021Pairwise Supervised Contrastive Learning of Sentence Representationscited by 31
  38. 2024Source Code Summarization in the Era of Large Language Modelscited by 31

50 most cited of 489 All 489 (JSON)