mlsanityData to 14 Oct 2026

Happiness is Sharing a Vocabulary: A Study of Transliteration Methods

arXiv 2510.10827 ↗First posted 12 Oct 2025cs.CL cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Transliteration has emerged as a promising means to bridge the gap between various languages in multilingual NLP, showing promising results especially for languages using non-Latin scripts. We investigate the degree to which shared script, overlapping token vocabularies, and shared phonology contribute to performance of multilingual models. To this end, we conduct controlled experiments using three kinds of transliteration (romanization, phonemic transcription, and substitution ciphers) as well as orthography. We evaluate each model on three downstream tasks -- named entity recognition (NER), part-of-speech tagging (POS) and natural language inference (NLI) -- and find that romanization significantly outperforms other input types in 11 out of 12 evaluation settings, largely consistent with our hypothesis that it is the most effective approach. We further analyze how each factor contributed to the success, and suggest that having longer (subword) tokens shared with pre-trained languages leads to better utilization of the model.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 18 of its 35 references that are in the map. First posted on arXiv 12 Oct 2025.

Builds on

18 of 35 references.

References · 18

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  3. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
  4. 2019How multilingual is Multilingual BERT?cited by 645
  5. 2020How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Modelscited by 318
  6. 2020Universal Dependencies v2: An Evergrowing Multilingual Treebank Collectioncited by 271
  7. 2020When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Modelscited by 131
  8. 2020UNKs Everywhere: Adapting Multilingual Language Models to New Scriptscited by 113
  9. 2023Tokenization Impacts Multilingual Language Modeling: Assessing Vocabulary Allocation and Overlap Across Languagescited by 62
  10. 2023Towards a Common Understanding of Contributing Factors for Cross-Lingual Transfer in Multilingual Language Models: A Reviewcited by 50
  11. 2024RomanSetu: Efficiently unlocking multilingual capabilities of Large Language Models via Romanizationcited by 29
  12. 2022Match the Script, Adapt if Multilingual: Analyzing the Effect of Multilingual Pretraining on Cross-lingual Transferabilitycited by 26
6 more
  1. 2023Romanization-based Large-scale Adaptation of Multilingual Language Modelscited by 21
  2. 2018Adapting Word Embeddings to New Languages with Morphological and Phonological Subword Representationscited by 18
  3. 2023The taste of IPA: Towards open-vocabulary keyword spotting and forced alignment in any languagecited by 14
  4. 2023Enhancing Cross-lingual Transfer via Phonemic Transcription Integrationcited by 11
  5. 2024Mitigating the Linguistic Gap with Phonemic Representations for Robust Cross-lingual Transfercited by 6
  6. 2024Prompting with Phonemes: Enhancing LLMs' Multilinguality for Non-Latin Script Languagescited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Linear Script Representations in Speech Foundation Models Enable Zero-Shot Transliterationcited by 0
  2. 2026One Form to Transfer Them All: Pretraining Multilingual Language Models Beyond Native Orthographycited by 0