mlsanityData to 14 Oct 2026

Multilingual Sentence Transformer as A Multilingual Word Aligner

arXiv 2301.12140 ↗First posted 28 Jan 2023cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

Multilingual pretrained language models (mPLMs) have shown their effectiveness in multilingual word alignment induction. However, these methods usually start from mBERT or XLM-R. In this paper, we investigate whether multilingual sentence Transformer LaBSE is a strong multilingual word aligner. This idea is non-trivial as LaBSE is trained to learn language-agnostic sentence-level embeddings, while the alignment extraction task requires the more fine-grained word-level embeddings to be language-agnostic. We demonstrate that the vanilla LaBSE outperforms other mPLMs currently used in the alignment task, and then propose to finetune LaBSE on parallel corpus for further improvement. Experiment results on seven language pairs show that our best aligner outperforms previous state-of-the-art models of all varieties. In addition, our aligner supports different language pairs in a single model, and even achieves new state-of-the-art on zero-shot language pairs that does not appear in the finetuning process.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 29 of its 53 references that are in the map. First posted on arXiv 28 Jan 2023.

Builds on

29 of 53 references.

References · 29

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  5. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  6. 2019Cross-lingual Language Model Pretrainingcited by 1,090
  7. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  8. 2020Beyond English-Centric Multilingual Machine Translationcited by 561
  9. 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
  10. 2019Simple, Scalable Adaptation for Neural Machine Translationcited by 242
  11. 2019Multilingual Universal Sentence Encoder for Semantic Retrievalcited by 193
  12. 2021Word Alignment by Fine-tuning Embeddings on Parallel Corporacited by 186
17 more
  1. 2020SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddingscited by 156
  2. 2021On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptationcited by 147
  3. 2018Rapid Adaptation of Neural Machine Translation to New Languagescited by 82
  4. 2019Jointly Learning to Align and Translate with Transformer Modelscited by 75
  5. 2020Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Informationcited by 64
  6. 2020Cross-Lingual Semantic Role Labeling with High-Quality Translated Training Corpuscited by 49
  7. 2017Weakly Supervised Cross-Lingual Named Entity Recognition via Effective Annotation and Representation Projectioncited by 36
  8. 2018Neural Machine Translation Decoding with Terminology Constraintscited by 36
  9. 2019Entity Projection via Machine Translation for Cross-Lingual NERcited by 32
  10. 2020Accurate Word Alignment Induction from Neural Machine Translationcited by 32
  11. 2021Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitutioncited by 31
  12. 2019Adding Interpretable Attention to Neural Translation Models Improves Word Alignmentcited by 30
  13. 2020End-to-End Neural Word Alignment Outperforms GIZA++cited by 28
  14. 2020Mask-Align: Self-Supervised Neural Word Alignmentcited by 20
  15. 2021Bilingual Lexicon Induction via Unsupervised Bitext Construction and Word Alignmentcited by 19
  16. 2014Contrastive Unsupervised Word Alignment with Non-Local Featurescited by 11
  17. 2021Graph Algorithms for Multiparallel Word Alignmentcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 3; 2026: 2.

8 papers.

  1. 2023To Translate or Not to Translate: A Systematic Investigation of Translation-Based Cross-Lingual Transfer to Low-Resource Languagescited by 7
  2. 2024BinaryAlign: Word Alignment as Binary Sequence Labelingcited by 3
  3. 2025The Devil Is in the Word Alignment Details: On Translation-Based Cross-Lingual Transfer for Token Classification Taskscited by 2
  4. 2024Cross-lingual Contextualized Phrase Retrievalcited by 1
  5. 2026OmniAlign: A Unified Multilingual Aligner for Word and Sentence Alignmentcited by 1
  6. 2026Scaling Unsupervised Word Alignment to Documents via Structural Constraintscited by 1
  7. 2025SMCLM: Semantically Meaningful Causal Language Modeling for Autoregressive Paraphrase Generationcited by 0
  8. 2025TransAlign: Machine Translation Encoders are Strong Word Aligners, Toocited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment
  2. 2024Modular Sentence Encoders: Separating Language Specialization from Cross-Lingual Alignment
  3. 2026Cross-Lingual Alignment Without Joint Training: Do Monolingual Language Models Converge on Universal Representations?
  4. 2022Cross-Align: Modeling Deep Cross-lingual Interactions for Word Alignment
  5. 2025AlignX: Advancing Multilingual Large Language Models with Multilingual Representation Alignment
  6. 2024Concept Space Alignment in Multilingual LLMs
  7. 2021XeroAlign: Zero-Shot Cross-lingual Transformer Alignment
  8. 2024Aligner: Efficient Alignment by Learning to Correct