mlsanityData to 14 Oct 2026

RETVec: Resilient and Efficient Text Vectorizer

arXiv 2302.09207 ↗First posted 18 Feb 2023cs.CL cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

This paper describes RETVec, an efficient, resilient, and multilingual text vectorizer designed for neural-based text processing. RETVec combines a novel character encoding with an optional small embedding model to embed words into a 256-dimensional vector space. The RETVec embedding model is pre-trained using pair-wise metric learning to be robust against typos and character-level adversarial attacks. In this paper, we evaluate and compare RETVec to state-of-the-art vectorizers and word embeddings on popular model architectures and datasets. These comparisons demonstrate that RETVec leads to competitive, multilingual models that are significantly more resilient to typos and adversarial text attacks. RETVec is available under the Apache 2 license at https://github.com/google-research/retvec.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 27 of its 43 references that are in the map. First posted on arXiv 18 Feb 2023.

Builds on

27 of 43 references.

References · 27

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  4. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  5. 2017Neural Discrete Representation Learningcited by 4,712
  6. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  7. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  8. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  9. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  10. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  11. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  12. 2016Enriching Word Vectors with Subword Informationcited by 1,583
15 more
  1. 2016Bag of Tricks for Efficient Text Classificationcited by 876
  2. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 723
  3. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 509
  4. 2020Circle Loss: A Unified Perspective of Pair Similarity Optimizationcited by 403
  5. 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 381
  6. 2019Multi-Similarity Loss with General Pair Weighting for Deep Metric Learningcited by 368
  7. 2018Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifierscited by 345
  8. 2018Learning Word Vectors for 157 Languagescited by 280
  9. 2022Transformer Quality in Linear Timecited by 191
  10. 2019Combating Adversarial Misspellings with Robust Word Recognitioncited by 163
  11. 2022MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languagescited by 146
  12. 2018Triplet-Center Loss for Multi-View 3D Object Retrievalcited by 69
  13. 2015Zero-Shot Learning via Joint Latent Similarity Embeddingcited by 39
  14. 2020NeuSpell: A Neural Spelling Correction Toolkitcited by 32
  15. 2021Dealing with Typos for BERT-based Passage Retrieval and Rankingcited by 23

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 0; 2025: 1.

3 papers.

  1. 2023A Survey on Malware Detection with Graph Representation Learningcited by 11
  2. 2023RETSim: Resilient and Efficient Text Similaritycited by 2
  3. 2025MultiFluxAI Enhancing Platform Engineering with Advanced Agent-Orchestrated Retrieval Systemscited by 0