# Robust Generalization Strategies for Morpheme Glossing in an Endangered Language Documentation Context

arXiv: https://arxiv.org/abs/2311.02777 · HTML: https://mlsanity.com/paper/2311.02777/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 32 references that are in the map.
- First posted on arXiv 5 Nov 2023.

## Builds on (most cited first, 7 of 7)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Self-Training for End-to-End Speech Recognition](https://mlsanity.com/paper/1909.09116/) (2019)
- [How Can We Accelerate Progress Towards Human-like Linguistic Generalization?](https://mlsanity.com/paper/2005.00955/) (2020)
- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [MicroBERT: Effective Training of Low-resource Monolingual BERTs through Parameter Reduction and Multitask Learning](https://mlsanity.com/paper/2212.12510/) (2022)
- [Cross-lingual Knowledge Transfer and Iterative Pseudo-labeling for Low-Resource Speech Recognition with Transducers](https://mlsanity.com/paper/2305.13652/) (2023)

## Built on by (most cited first, 4 of 4)

- [Can LLMs Really Learn to Translate a Low-Resource Language from One Grammar Book?](https://mlsanity.com/paper/2409.19151/) (2024)
- [Can we teach language models to gloss endangered languages?](https://mlsanity.com/paper/2406.18895/) (2024)
- [GlossLM: A Massively Multilingual Corpus and Pretrained Model for Interlinear Glossed Text](https://mlsanity.com/paper/2403.06399/) (2024)
- [NLP Progress in Indigenous Latin American Languages](https://mlsanity.com/paper/2404.05365/) (2024)

## Abstract (arXiv)

Generalization is of particular importance in resource-constrained settings, where the available training data may represent only a small fraction of the distribution of possible texts. We investigate the ability of morpheme labeling models to generalize by evaluating their performance on unseen genres of text, and we experiment with strategies for closing the gap between performance on in-distribution and out-of-distribution data. Specifically, we use weight decay optimization, output denoising, and iterative pseudo-labeling, and achieve a 2% improvement on a test set containing texts from unseen genres. All experiments are performed using texts written in the Mayan language Uspanteko.