mlsanityData to 7 Oct 2026

Enhancing Cross-lingual Sentence Embedding for Low-resource Languages with Word Alignment

arXiv 2404.02490 ↗First posted 3 Apr 2024cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

The field of cross-lingual sentence embeddings has recently experienced significant advancements, but research concerning low-resource languages has lagged due to the scarcity of parallel corpora. This paper shows that cross-lingual word representation in low-resource languages is notably under-aligned with that in high-resource languages in current models. To address this, we introduce a novel framework that explicitly aligns words between English and eight low-resource languages, utilizing off-the-shelf word alignment models. This framework incorporates three primary training objectives: aligned word prediction and word translation ranking, along with the widely used translation ranking. We evaluate our approach through experiments on the bitext retrieval task, which demonstrate substantial improvements on sentence embeddings in low-resource languages. In addition, the competitive performance of the proposed model across a broader range of tasks in high-resource languages underscores its practicality.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 23 of its 36 references that are in the map. First posted on arXiv 3 Apr 2024.

Builds on

23 of 36 references.

References · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  4. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  5. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  6. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  7. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  8. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  9. 2020Language-agnostic BERT Sentence Embeddingcited by 621
  10. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  11. 2020InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Trainingcited by 237
  12. 2021Word Alignment by Fine-tuning Embeddings on Parallel Corporacited by 186
11 more
  1. 2020SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddingscited by 155
  2. 2019Improving Multilingual Sentence Embedding using Bi-directional Dual Encoder with Additive Margin Softmaxcited by 58
  3. 2021Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignmentcited by 41
  4. 2018Effective Parallel Corpus Mining using Bilingual Sentence Embeddingscited by 39
  5. 2022PCL: Peer-Contrastive Learning with Diverse Augmentations for Unsupervised Sentence Embeddingscited by 29
  6. 2020A Supervised Word Alignment Method based on Cross-Language Span Prediction using Multilingual BERTcited by 27
  7. 2022English Contrastive Learning Can Learn Universal Cross-lingual Sentence Embeddingscited by 26
  8. 2023WSPAlign: Word Alignment Pre-training via Large-Scale Weakly Supervised Span Predictioncited by 10
  9. 2023Dual-Alignment Pre-training for Cross-lingual Sentence Embeddingcited by 9
  10. 2023Leveraging Multi-lingual Positive Instances in Contrastive Learning to Improve Sentence Embeddingcited by 9
  11. 2023VECO 2.0: Cross-lingual Language Model Pre-training with Multi-granularity Contrastive Learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 4; 2026: 5.

13 papers.

Sort:

By area:

  1. 2024Word Alignment as Preference for Machine Translationcited by 14
  2. 2025Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaningcited by 10
  3. 2024Robust and Minimally Invasive Watermarking for EaaScited by 4
  4. 2025Conan-Embedding-v2: Training an LLM from Scratch for Text Embeddingscited by 4
  5. 2026NeoAMT: Neologism-Aware Agentic Machine Translation with Reinforcement Learningcited by 3
  6. 2024Space Decomposition for Sentence Embeddingcited by 1
  7. 2026Bridging Linguistic Gaps: Cross-Lingual Mapping in Pre-Training and Dataset for Enhanced Multilingual LLM Performancecited by 1
  8. 2026MetaHOPE: A Metaphor-Oriented Evaluation Framework for Analysing MT and LLM Translation Errorscited by 1
  9. 2026Scaling Unsupervised Word Alignment to Documents via Structural Constraintscited by 1
  10. 2024Unsupervised Bilingual Lexicon Induction for Low Resource Languagescited by 0
  11. 2025LexPro-1.0 Technical Reportcited by 0
  12. 2025CUTE: A Multilingual Dataset for Enhancing Cross-Lingual Knowledge Transfer in Low-Resource Languagescited by 0
1 more
  1. 2026GRC: Unifying Reasoning-Driven Generation, Retrieval and Compressioncited by 0