mlsanityData to 14 Oct 2026

Learning Multilingual Sentence Representations with Cross-lingual Consistency Regularization

arXiv 2306.06919 ↗First posted 12 Jun 2023cs.CL cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Multilingual sentence representations are the foundation for similarity-based bitext mining, which is crucial for scaling multilingual neural machine translation (NMT) system to more languages. In this paper, we introduce MuSR: a one-for-all Multilingual Sentence Representation model that supports more than 220 languages. Leveraging billions of English-centric parallel corpora, we train a multilingual Transformer encoder, coupled with an auxiliary Transformer decoder, by adopting a multilingual NMT framework with CrossConST, a cross-lingual consistency regularization technique proposed in Gao et al. (2023). Experimental results on multilingual similarity search and bitext mining tasks show the effectiveness of our approach. Specifically, MuSR achieves superior performance over LASER3 (Heffernan et al., 2022) which consists of 148 independent multilingual sentence encoders.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 23 of its 30 references that are in the map. First posted on arXiv 12 Jun 2023.

Builds on

23 of 30 references.

References · 23

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  3. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  4. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  5. 2019Cross-lingual Language Model Pretrainingcited by 1,090
  6. 2016Bag of Tricks for Efficient Text Classificationcited by 876
  7. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  8. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 593
  9. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  10. 2016FastText.zip: Compressing text classification modelscited by 335
  11. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 257
  12. 2019Simple, Scalable Adaptation for Neural Machine Translationcited by 242
11 more
  1. 2019WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipediacited by 223
  2. 2019Multilingual Universal Sentence Encoder for Semantic Retrievalcited by 193
  3. 2019CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEBcited by 177
  4. 2019CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairscited by 136
  5. 2018Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddingscited by 83
  6. 2019Improving Multilingual Sentence Embedding using Bi-directional Dual Encoder with Additive Margin Softmaxcited by 58
  7. 2022Bitext Mining Using Distilled Sentence Representations for Low-Resource Languagescited by 48
  8. 2018Dual Conditional Cross-Entropy Filtering of Noisy Parallel Corporacited by 40
  9. 2018Effective Parallel Corpus Mining using Bilingual Sentence Embeddingscited by 39
  10. 2023LEALLA: Learning Lightweight Language-agnostic Sentence Embeddings with Knowledge Distillationcited by 11
  11. 2023Improving Zero-shot Multilingual Neural Machine Translation by Leveraging Cross-lingual Consistency Regularizationcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 3; 2026: 1.

7 papers.

  1. 2025Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Studycited by 38
  2. 2024IRCoder: Intermediate Representations Make Language Models Robust Multilingual Code Generatorscited by 23
  3. 2024Towards Boosting Many-to-Many Multilingual Machine Translation with Large Language Modelscited by 5
  4. 2024Modular Sentence Encoders: Separating Language Specialization from Cross-Lingual Alignmentcited by 2
  5. 2026Explainable Disentangled Representation Learning for Generalizable Authorship Attribution in the Era of Generative AIcited by 1
  6. 2025LANGALIGN: Enhancing Non-English Language Models via Cross-Lingual Embedding Alignmentcited by 0
  7. 2025A Systematic Analysis of Base Model Choice for Reward Modelingcited by 0