mlsanityData to 14 Oct 2026

Dictionaries to the Rescue: Cross-Lingual Vocabulary Transfer for Low-Resource Languages Using Bilingual Dictionaries

arXiv 2506.01535 ↗First posted 2 Jun 2025cs.CL cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Cross-lingual vocabulary transfer plays a promising role in adapting pre-trained language models to new languages, including low-resource languages. Existing approaches that utilize monolingual or parallel corpora face challenges when applied to languages with limited resources. In this work, we propose a simple yet effective vocabulary transfer method that utilizes bilingual dictionaries, which are available for many languages, thanks to descriptive linguists. Our proposed method leverages a property of BPE tokenizers where removing a subword from the vocabulary causes a fallback to shorter subwords. The embeddings of target subwords are estimated iteratively by progressively removing them from the tokenizer. The experimental results show that our approach outperforms existing methods for low-resource languages, demonstrating the effectiveness of a dictionary-based approach for cross-lingual vocabulary transfer.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 22 of its 40 references that are in the map. First posted on arXiv 2 Jun 2025.

Builds on

22 of 40 references.

References · 22

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  3. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  4. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  5. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  6. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  7. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 360
  8. 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 273
  9. 2023Language Model Tokenizers Introduce Unfairness Between Languagescited by 214
  10. 2023Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Modelscited by 192
  11. 2021WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language modelscited by 89
  12. 2020Extending Multilingual BERT to Low-Resource Languagescited by 88
10 more
  1. 2023OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient Large-scale Multilingual Continued Pretrainingcited by 37
  2. 2024Trans-Tokenization and Cross-lingual Vocabulary Transfers: Language Adaptation of LLMs for Low-Resource NLPcited by 36
  3. 2025MorphBPE: A Morpho-Aware Tokenizer Bridging Linguistic Complexity for Efficient LLM Training Across Morphologiescited by 25
  4. 2024An Empirical Study on Cross-lingual Vocabulary Adaptation for Efficient Language Model Inferencecited by 18
  5. 2024Lexically Grounded Subword Segmentationcited by 13
  6. 2017Model Transfer for Tagging Low-resource Languages using a Bilingual Dictionarycited by 10
  7. 2019Lost in Evaluation: Misleading Benchmarks for Bilingual Dictionary Inductioncited by 10
  8. 2024Adapters for Altering LLM Vocabularies: What Languages Benefit the Most?cited by 9
  9. 2020Unigram-Normalized Perplexity as a Language Model Performance Measure with Different Vocabulary Sizescited by 6
  10. 2025Optimizing LLMs for Italian: Reducing Token Fertility and Enhancing Efficiency Through Vocabulary Adaptationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Defragmenting Language Models: An Interpretability-based Approach for Vocabulary Expansioncited by 2
  2. 2026AFRILANGTUTOR: Advancing Language Tutoring and Culture Education in Low-Resource Languages with Large Language Modelscited by 1