mlsanityData to 14 Oct 2026

Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation

arXiv 2004.09813 ↗First posted 21 Apr 2020cs.CLcited by 593

Not on the map. Why?

Cited by 23 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present an easy and efficient method to extend existing sentence embedding models to new languages. This allows to create multilingual versions from previously monolingual models. The training is based on the idea that a translated sentence should be mapped to the same location in the vector space as the original sentence. We use the original (monolingual) model to generate sentence embeddings for the source language and then train a new system on translated sentences to mimic the original model. Compared to other methods for training multilingual sentence embeddings, this approach has several advantages: It is easy to extend existing models with relatively few samples to new languages, it is easier to ensure desired properties for the vector space, and the hardware requirements for training is lower. We demonstrate the effectiveness of our approach for 50+ languages from various language families. Code to extend sentence embeddings models to more than 400 languages is publicly available.

Key facts. Cited by 593 papers in the map (data through 14 Oct 2026). First posted on arXiv 21 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 62; 2022: 88; 2023: 116; 2024: 139; 2025: 108; 2026: 80.

593 papers.

Sort:

By area:

  1. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  2. 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 801
  3. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
  4. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 463
  5. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 371
  6. 2022UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Modelscited by 231
  7. 2024Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilitiescited by 186
  8. 2024Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrencecited by 159
  9. 2021Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languagescited by 136
  10. 2024CinePile: A Long Video Question Answering Dataset and Benchmarkcited by 114
  11. 2023A Semantic Invariant Robust Watermark for Large Language Modelscited by 112
  12. 2024MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Datacited by 109
38 more
  1. 2023SONAR: Sentence-Level Multimodal and Language-Agnostic Representationscited by 108
  2. 2024The Art of Saying No: Contextual Noncompliance in Language Modelscited by 105
  3. 2025SEED-GRPO: Semantic Entropy Enhanced GRPO for Uncertainty-Aware Policy Optimizationcited by 89
  4. 2022Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Surveycited by 80
  5. 2023LLMaAA: Making Large Language Models as Active Annotatorscited by 79
  6. 2023Multilingual LLMs are Better Cross-lingual In-context Learners with Alignmentcited by 71
  7. 2021Controlled Text Generation as Continuous Optimization with Multiple Constraintscited by 70
  8. 2024LLMs Are Few-Shot In-Context Low-Resource Language Learnerscited by 68
  9. 2024MoRAL: MoE Augmented LoRA for LLMs' Lifelong Learningcited by 63
  10. 2024GuardT2I: Defending Text-to-Image Models from Adversarial Promptscited by 62
  11. 2022UnitY: Two-pass Direct Speech-to-speech Translation with Discrete Unitscited by 60
  12. 2023Are LLMs All You Need for Task-Oriented Dialogue?cited by 54
  13. 2023Multilingual Language Models are not Multicultural: A Case Study in Emotioncited by 54
  14. 2022Bitext Mining Using Distilled Sentence Representations for Low-Resource Languagescited by 48
  15. 2021Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedbackcited by 47
  16. 2022Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentationcited by 44
  17. 2024Separating Tongue from Thought: Activation Patching Reveals Language-Agnostic Concept Representations in Transformerscited by 44
  18. 2025Rethinking Reflection in Pre-Trainingcited by 43
  19. 2025The Price of Format: Diversity Collapse in LLMscited by 43
  20. 2021Claim Matching Beyond English to Scale Global Fact-Checkingcited by 42
  21. 2023Amazon-M2: A Multilingual Multi-locale Shopping Session Dataset for Recommendation and Text Generationcited by 42
  22. 2024Segment Any Text: A Universal Approach for Robust, Efficient and Adaptable Sentence Segmentationcited by 41
  23. 2022EUR-Lex-Sum: A Multi- and Cross-lingual Dataset for Long-form Summarization in the Legal Domaincited by 40
  24. 2024LeDex: Training LLMs to Better Self-Debug and Explain Codecited by 40
  25. 2023MuAViC: A Multilingual Audio-Visual Corpus for Robust Speech Recognition and Robust Speech-to-Text Translationcited by 39
  26. 2024Benchmarking Linguistic Diversity of Large Language Modelscited by 38
  27. 2021Multilingual and Cross-Lingual Intent Detection from Spoken Datacited by 35
  28. 2022SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translationscited by 35
  29. 2022Speaking Multiple Languages Affects the Moral Bias of Language Modelscited by 34
  30. 2023TCRA-LLM: Token Compression Retrieval Augmented Large Language Model for Inference Cost Reductioncited by 34
  31. 2023MacGyver: Are Large Language Models Creative Problem Solvers?cited by 34
  32. 2024Understanding Cross-Lingual Alignment -- A Surveycited by 34
  33. 2022SBERT studies Meaning Representations: Decomposing Sentence Embeddings into Explainable Semantic Featurescited by 33
  34. 2025Training Sparse Mixture Of Experts Text Embedding Modelscited by 32
  35. 2022Quality Controlled Paraphrase Generationcited by 31
  36. 2021Learning Domain-Specialised Representations for Cross-Lingual Biomedical Entity Linkingcited by 28
  37. 2025Proactive Assistant Dialogue Generation from Streaming Egocentric Videoscited by 28
  38. 2025PromptLocate: Localizing Prompt Injection Attackscited by 28

50 most cited of 593 All 593 (JSON)