# Massively Multilingual Joint Segmentation and Glossing

arXiv: https://arxiv.org/abs/2601.10925 · HTML: https://mlsanity.com/paper/2601.10925/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 43 references that are in the map.
- First posted on arXiv 16 Jan 2026.

## Builds on (most cited first, 15 of 15)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [ByT5: Towards a token-free future with pre-trained byte-to-byte models](https://mlsanity.com/paper/2105.13626/) (2021)
- [Can LLMs Really Learn to Translate a Low-Resource Language from One Grammar Book?](https://mlsanity.com/paper/2409.19151/) (2024)
- [Can we teach language models to gloss endangered languages?](https://mlsanity.com/paper/2406.18895/) (2024)
- [GlossLM: A Massively Multilingual Corpus and Pretrained Model for Interlinear Glossed Text](https://mlsanity.com/paper/2403.06399/) (2024)
- [Tackling the Low-resource Challenge for Canonical Segmentation](https://mlsanity.com/paper/2010.02804/) (2020)
- [Wav2Gloss: Generating Interlinear Glossed Text from Speech](https://mlsanity.com/paper/2403.13169/) (2024)
- [LingGym: How Far Are LLMs from Thinking Like Field Linguists?](https://mlsanity.com/paper/2511.00343/) (2025)
- [Using Interlinear Glosses as Pivot in Low-Resource Multilingual Machine Translation](https://mlsanity.com/paper/1911.02709/) (2019)
- [Multiple Sources are Better Than One: Incorporating External Knowledge in Low-Resource Glossing](https://mlsanity.com/paper/2406.11085/) (2024)
- [GrammaMT: Improving Machine Translation with Grammar-Informed In-Context Learning](https://mlsanity.com/paper/2410.18702/) (2024)
- [User-Centric Evaluation of OCR Systems for Kwak'wala](https://mlsanity.com/paper/2302.13410/) (2023)
- [Interdisciplinary Research in Conversation: A Case Study in Computational Morphology for Language Documentation](https://mlsanity.com/paper/2509.10644/) (2025)
- [Teacher Perception of Automatically Extracted Grammar Concepts for L2 Language Learning](https://mlsanity.com/paper/2206.05154/) (2022)

## Built on by (most cited first, 1 of 1)

- [CWoMP: Morpheme Representation Learning for Interlinear Glossing](https://mlsanity.com/paper/2603.18184/) (2026)

## Abstract (arXiv)

Automated interlinear gloss prediction with neural networks is a promising approach to accelerate language documentation efforts. However, while state-of-the-art models like GlossLM achieve high scores on glossing benchmarks, user studies with linguists have found critical barriers to the usefulness of such models in real-world scenarios. In particular, existing models typically generate morpheme-level glosses but assign them to whole words without predicting the actual morpheme boundaries, making the predictions less interpretable and thus untrustworthy to human annotators. We conduct the first study on neural models that jointly predict interlinear glosses and the corresponding morphological segmentation from raw text. We run experiments to determine the optimal way to train models that balance segmentation and glossing accuracy, as well as the alignment between the two tasks. We extend the training corpus of GlossLM and pretrain PolyGloss, a family of seq2seq multilingual models for joint segmentation and glossing that outperforms GlossLM on glossing and beats various open-source LLMs on segmentation, glossing, and alignment. In addition, we demonstrate that PolyGloss can be quickly adapted to a new dataset via low-rank adaptation.