mlsanityData to 14 Oct 2026

Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond

arXiv 1812.10464 ↗First posted 26 Dec 2018cs.CL cs.AI cs.LGcited by 434

Not on the map. Why?

Abstract (arXiv)

We introduce an architecture to learn joint multilingual sentence representations for 93 languages, belonging to more than 30 different families and written in 28 different scripts. Our system uses a single BiLSTM encoder with a shared BPE vocabulary for all languages, which is coupled with an auxiliary decoder and trained on publicly available parallel corpora. This enables us to learn a classifier on top of the resulting embeddings using English annotated data only, and transfer it to any of the 93 languages without any modification. Our experiments in cross-lingual natural language inference (XNLI dataset), cross-lingual document classification (MLDoc dataset) and parallel corpus mining (BUCC dataset) show the effectiveness of our approach. We also introduce a new test set of aligned sentences in 112 languages, and show that our sentence embeddings obtain strong results in multilingual similarity search even for low-resource languages. Our implementation, the pre-trained encoder and the multilingual test set are available at https://github.com/facebookresearch/LASER

Key facts. Cited by 434 papers in the map (data through 14 Oct 2026). First posted on arXiv 26 Dec 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 101; 2022: 77; 2023: 83; 2024: 80; 2025: 53; 2026: 40.

434 papers.

Sort:

By area:

  1. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  2. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  3. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 463
  4. 2024Multilingual E5 Text Embeddings: A Technical Reportcited by 372
  5. 2023Improving Text Embeddings with Large Language Modelscited by 303
  6. 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 243
  7. 2021XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluationcited by 168
  8. 2021Constrained Language Models Yield Few-Shot Semantic Parserscited by 163
  9. 2021The Curious Case of Hallucinations in Neural Machine Translationcited by 161
  10. 2021Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languagescited by 136
  11. 2021BanglaBERT: Language Model Pretraining and Benchmarks for Low-Resource Language Understanding Evaluation in Banglacited by 125
  12. 2021mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Datasetcited by 116
38 more
  1. 2023Glot500: Scaling Multilingual Corpora and Language Models to 500 Languagescited by 116
  2. 2021Contrastive Learning for Many-to-many Multilingual Neural Machine Translationcited by 108
  3. 2023SONAR: Sentence-Level Multimodal and Language-Agnostic Representationscited by 108
  4. 2021The Multilingual TEDx Corpus for Speech Recognition and Translationcited by 101
  5. 2021Survey of Low-Resource Machine Translationcited by 92
  6. 2021Language Models are Few-shot Multilingual Learnerscited by 92
  7. 2021MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfercited by 90
  8. 2022BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Promptingcited by 90
  9. 2024Large Concept Models: Language Modeling in a Sentence Representation Spacecited by 89
  10. 2022Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Bettercited by 75
  11. 2024Multilingual Instruction Tuning With Just a Pinch of Multilingualitycited by 73
  12. 2021XLM-E: Cross-lingual Language Model Pre-training via ELECTRAcited by 72
  13. 2023Evaluating GPT-4 and ChatGPT on Japanese Medical Licensing Examinationscited by 63
  14. 2023Tokenization Impacts Multilingual Language Modeling: Assessing Vocabulary Allocation and Overlap Across Languagescited by 62
  15. 2022Gender Bias in Masked Language Models for Multiple Languagescited by 60
  16. 2023Monolingual or Multilingual Instruction Tuning: Which Makes a Better Alpacacited by 60
  17. 2021How to Adapt Your Pretrained Multilingual Model to 1600 Languagescited by 54
  18. 2023LLM-powered Data Augmentation for Enhanced Cross-lingual Performancecited by 51
  19. 2021AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languagescited by 50
  20. 2021A Primer on Pretrained Multilingual Language Modelscited by 49
  21. 2021Discrete and Soft Prompting for Multilingual Modelscited by 49
  22. 2022Bitext Mining Using Distilled Sentence Representations for Low-Resource Languagescited by 48
  23. 2023Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspectioncited by 44
  24. 2021Claim Matching Beyond English to Scale Global Fact-Checkingcited by 42
  25. 2023HateMM: A Multi-Modal Dataset for Hate Video Classificationcited by 42
  26. 2021MT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairscited by 41
  27. 2021CrossSum: Beyond English-Centric Cross-Lingual Summarization for 1,500+ Language Pairscited by 39
  28. 2023OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient Large-scale Multilingual Continued Pretrainingcited by 37
  29. 2022SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translationscited by 35
  30. 2023Lost in Translation: Large Language Models in Non-English Content Analysiscited by 35
  31. 2024A Shocking Amount of the Web is Machine Translated: Insights from Multi-Way Parallelismcited by 35
  32. 2024Understanding Cross-Lingual Alignment -- A Surveycited by 34
  33. 2022SAMU-XLSR: Semantically-Aligned Multimodal Utterance-level Cross-Lingual Speech Representationcited by 32
  34. 2024Preference Tuning For Toxicity Mitigation Generalizes Across Languagescited by 31
  35. 2021JobBERT: Understanding Job Titles through Skillscited by 29
  36. 2022Findings of the The RuATD Shared Task 2022 on Artificial Text Detection in Russiancited by 29
  37. 2025Middle-Layer Representation Alignment for Cross-Lingual Transfer in Fine-Tuned LLMscited by 29
  38. 2021Learning Domain-Specialised Representations for Cross-Lingual Biomedical Entity Linkingcited by 28

50 most cited of 434 All 434 (JSON)