# Adapting Word Embeddings to New Languages with Morphological and Phonological Subword Representations

arXiv: https://arxiv.org/abs/1808.09500 · HTML: https://mlsanity.com/paper/1808.09500/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Aug 2018.

## Built on by (most cited first, 18 of 18)

- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [Machine Translation into Low-resource Language Varieties](https://mlsanity.com/paper/2106.06797/) (2021)
- [Does Transliteration Help Multilingual Language Modeling?](https://mlsanity.com/paper/2201.12501/) (2022)
- [HiNER: A Large Hindi Named Entity Recognition Dataset](https://mlsanity.com/paper/2204.13743/) (2022)
- [Enhancing Cross-lingual Transfer via Phonemic Transcription Integration](https://mlsanity.com/paper/2307.04361/) (2023)
- [Sub-Character Tokenization for Chinese Pretrained Language Models](https://mlsanity.com/paper/2106.00400/) (2021)
- [PWESuite: Phonetic Word Embeddings and Tasks They Facilitate](https://mlsanity.com/paper/2304.02541/) (2023)
- [Mitigating the Linguistic Gap with Phonemic Representations for Robust Cross-lingual Transfer](https://mlsanity.com/paper/2402.14279/) (2024)
- [A Morphology-Based Investigation of Positional Encodings](https://mlsanity.com/paper/2404.04530/) (2024)
- [Data-adaptive Transfer Learning for Translation: A Case Study in Haitian and Jamaican](https://mlsanity.com/paper/2209.06295/) (2022)
- [Happiness is Sharing a Vocabulary: A Study of Transliteration Methods](https://mlsanity.com/paper/2510.10827/) (2025)
- [Search Intenion Network for Personalized Query Auto-Completion in E-Commerce](https://mlsanity.com/paper/2403.02609/) (2024)
- [Cross-Lingual IPA Contrastive Learning for Zero-Shot NER](https://mlsanity.com/paper/2503.07214/) (2025)
- [Evaluation of Morphological Embeddings for the Russian Language](https://mlsanity.com/paper/2103.06628/) (2021)
- [Evaluation of Morphological Embeddings for English and Russian Languages](https://mlsanity.com/paper/2103.06884/) (2021)
- [Korean Bio-Medical Corpus (KBMC) for Medical Named Entity Recognition](https://mlsanity.com/paper/2403.16158/) (2024)
- [Zero-Shot Cross-Lingual NER Using Phonemic Representations for Low-Resource Languages](https://mlsanity.com/paper/2406.16030/) (2024)
- [CoPiT: Cognitive Pivot Translation for Digraphic Low-Resource Mongolian in the Traditional Script](https://mlsanity.com/paper/2607.05849/) (2026)

## Abstract (arXiv)

Much work in Natural Language Processing (NLP) has been for resource-rich languages, making generalization to new, less-resourced languages challenging. We present two approaches for improving generalization to low-resourced languages by adapting continuous word representations using linguistically motivated subword units: phonemes, morphemes and graphemes. Our method requires neither parallel corpora nor bilingual dictionaries and provides a significant gain in performance over previous methods relying on these resources. We demonstrate the effectiveness of our approaches on Named Entity Recognition for four languages, namely Uyghur, Turkish, Bengali and Hindi, of which Uyghur and Bengali are low resource languages, and also perform experiments on Machine Translation. Exploiting subwords with transfer learning gives us a boost of +15.2 NER F1 for Uyghur and +9.7 F1 for Bengali. We also show improvements in the monolingual setting where we achieve (avg.) +3 F1 and (avg.) +1.35 BLEU.