# On the Importance of Subword Information for Morphological Tasks in Truly Low-Resource Languages

arXiv: https://arxiv.org/abs/1909.12375 · HTML: https://mlsanity.com/paper/1909.12375/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Sep 2019.

## Built on by (most cited first, 11 of 11)

- [Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Ahead](https://mlsanity.com/paper/2505.21315/) (2025)
- [Subword Segmental Language Modelling for Nguni Languages](https://mlsanity.com/paper/2210.06525/) (2022)
- [Evaluating Subword Tokenization: Alien Subword Composition and OOV Generalization Challenge](https://mlsanity.com/paper/2404.13292/) (2024)
- [Subword Segmental Machine Translation: Unifying Segmentation and Target Sentence Generation](https://mlsanity.com/paper/2305.07005/) (2023)
- [An Assessment of the Impact of OCR Noise on Language Models](https://mlsanity.com/paper/2202.00470/) (2022)
- [Triples-to-isiXhosa (T2X): Addressing the Challenges of Low-Resource Agglutinative Data-to-Text Generation](https://mlsanity.com/paper/2403.07567/) (2024)
- [Tokenization Disparities as Infrastructure Bias: How Subword Systems Create Inequities in LLM Access and Efficiency](https://mlsanity.com/paper/2510.12389/) (2025)
- [The Learning Dynamics of Subword Segmentation for Morphologically Diverse Languages](https://mlsanity.com/paper/2511.09197/) (2025)
- [Combining Deep Generative Models and Multi-lingual Pretraining for Semi-supervised Document Classification](https://mlsanity.com/paper/2101.10717/) (2021)
- [High-Resource Methodological Bias in Low-Resource Investigations](https://mlsanity.com/paper/2211.07534/) (2022)
- [What Tokens are Learned when Tokenization is Optimized Jointly with Language Modeling?](https://mlsanity.com/paper/2608.17325/) (2026)

## Abstract (arXiv)

Recent work has validated the importance of subword information for word representation learning. Since subwords increase parameter sharing ability in neural models, their value should be even more pronounced in low-data regimes. In this work, we therefore provide a comprehensive analysis focused on the usefulness of subwords for word representation learning in truly low-resource scenarios and for three representative morphological tasks: fine-grained entity typing, morphological tagging, and named entity recognition. We conduct a systematic study that spans several dimensions of comparison: 1) type of data scarcity which can stem from the lack of task-specific training data, or even from the lack of unannotated data required to train word embeddings, or both; 2) language type by working with a sample of 16 typologically diverse languages including some truly low-resource ones (e.g. Rusyn, Buryat, and Zulu); 3) the choice of the subword-informed word representation method. Our main results show that subword-informed models are universally useful across all language types, with large gains over subword-agnostic embeddings. They also suggest that the effective use of subwords largely depends on the language (type) and the task at hand, as well as on the amount of available data for training the embeddings and task-based models, where having sufficient in-task data is a more critical requirement.