# Tackling the Low-resource Challenge for Canonical Segmentation

arXiv: https://arxiv.org/abs/2010.02804 · HTML: https://mlsanity.com/paper/2010.02804/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Oct 2020.

## Built on by (most cited first, 10 of 10)

- [Not always about you: Prioritizing community needs when developing endangered language technology](https://mlsanity.com/paper/2204.05541/) (2022)
- [BPE vs. Morphological Segmentation: A Case Study on Machine Translation of Four Polysynthetic Languages](https://mlsanity.com/paper/2203.08954/) (2022)
- [The Zeno's Paradox of `Low-Resource' Languages](https://mlsanity.com/paper/2410.20817/) (2024)
- [Neural Machine Translation for the Indigenous Languages of the Americas: An Introduction](https://mlsanity.com/paper/2306.06804/) (2023)
- [Morphological Processing of Low-Resource Languages: Where We Are and What's Next](https://mlsanity.com/paper/2203.08909/) (2022)
- [NLP Progress in Indigenous Latin American Languages](https://mlsanity.com/paper/2404.05365/) (2024)
- [The Effect of Data Partitioning Strategy on Model Generalizability: A Case Study of Morphological Segmentation](https://mlsanity.com/paper/2404.09371/) (2024)
- [Massively Multilingual Joint Segmentation and Glossing](https://mlsanity.com/paper/2601.10925/) (2026)
- [TAMS: Translation-Assisted Morphological Segmentation](https://mlsanity.com/paper/2403.14840/) (2024)
- [CWoMP: Morpheme Representation Learning for Interlinear Glossing](https://mlsanity.com/paper/2603.18184/) (2026)

## Abstract (arXiv)

Canonical morphological segmentation consists of dividing words into their standardized morphemes. Here, we are interested in approaches for the task when training data is limited. We compare model performance in a simulated low-resource setting for the high-resource languages German, English, and Indonesian to experiments on new datasets for the truly low-resource languages Popoluca and Tepehua. We explore two new models for the task, borrowing from the closely related area of morphological generation: an LSTM pointer-generator and a sequence-to-sequence model with hard monotonic attention trained with imitation learning. We find that, in the low-resource setting, the novel approaches outperform existing ones on all languages by up to 11.4% accuracy. However, while accuracy in emulated low-resource scenarios is over 50% for all languages, for the truly low-resource languages Popoluca and Tepehua, our best model only obtains 37.4% and 28.4% accuracy, respectively. Thus, we conclude that canonical segmentation is still a challenging task for low-resource languages.