mlsanityData to 14 Oct 2026

UniBridge: A Unified Approach to Cross-Lingual Transfer Learning for Low-Resource Languages

arXiv 2406.09717 ↗First posted 14 Jun 2024cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

In this paper, we introduce UniBridge (Cross-Lingual Transfer Learning with Optimized Embeddings and Vocabulary), a comprehensive approach developed to improve the effectiveness of Cross-Lingual Transfer Learning, particularly in languages with limited resources. Our approach tackles two essential elements of a language model: the initialization of embeddings and the optimal vocabulary size. Specifically, we propose a novel embedding initialization method that leverages both lexical and semantic alignment for a language. In addition, we present a method for systematically searching for the optimal vocabulary size, ensuring a balance between model complexity and linguistic coverage. Our experiments across multilingual datasets show that our approach greatly improves the F1-Score in several languages. UniBridge is a robust and adaptable solution for cross-lingual systems in various languages, highlighting the significance of initializing embeddings and choosing the right vocabulary size in cross-lingual environments.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 34 of its 48 references that are in the map. First posted on arXiv 14 Jun 2024.

Builds on

34 of 48 references.

References · 34

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  3. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  4. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,475
  5. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  6. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
  7. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 742
  8. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 578
  9. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  10. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 445
  11. 2016From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classificationcited by 341
  12. 2020Are All Languages Created Equal in Multilingual BERT?cited by 207
22 more
  1. 2023Continual Pre-training of Language Modelscited by 198
  2. 2019BPE-Dropout: Simple and Effective Subword Regularizationcited by 195
  3. 2021Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Littlecited by 177
  4. 2022Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuningcited by 149
  5. 2022mGPT: Few-Shot Learners Go Multilingualcited by 142
  6. 2021Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillationcited by 134
  7. 2020The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MTcited by 128
  8. 2020UNKs Everywhere: Adapting Multilingual Language Models to New Scriptscited by 113
  9. 2021WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language modelscited by 89
  10. 2020Extending Multilingual BERT to Low-Resource Languagescited by 88
  11. 2023XLM-V: Overcoming the Vocabulary Bottleneck in Multilingual Masked Language Modelscited by 88
  12. 2020UDapter: Language Adaptation for Truly Universal Dependency Parsingcited by 77
  13. 2020Vocabulary Learning via Optimal Transport for Neural Machine Translationcited by 58
  14. 2021How to Adapt Your Pretrained Multilingual Model to 1600 Languagescited by 54
  15. 2022AfroLM: A Self-Active Learning-based Multilingual Pretrained Language Model for 23 African Languagescited by 46
  16. 2017Learning Language Representations for Typology Predictioncited by 40
  17. 2021Allocating Large Vocabulary Capacity for Cross-lingual Language Model Pre-trainingcited by 22
  18. 2024The Hidden Space of Transformer Language Adapterscited by 17
  19. 2022Prompt-Tuning Can Be Much Better Than Fine-Tuning on Cross-lingual Understanding With Multilingual Language Modelscited by 16
  20. 2021Subword Mapping and Anchoring across Languagescited by 12
  21. 2024LAMPAT: Low-Rank Adaption for Multilingual Paraphrasing Using Adversarial Trainingcited by 8
  22. 2024The Impact of Language Adapters in Cross-Lingual Transfer for NLUcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2026An Efficient and Effective Evaluator for Text2SQL Models on Unseen and Unlabeled Datacited by 2
  2. 2026Learning to Evaluate: Cost-Effective Model Evaluation on Unlabeled Data with Meta-Learningcited by 1
  3. 2025ALOPE: Adaptive Layer Optimization for Translation Quality Estimation using Large Language Modelscited by 0
  4. 2026An Efficient and Effective Agentic Group Shilling Attack on Recommender Systemscited by 0