# CWoMP: Morpheme Representation Learning for Interlinear Glossing

arXiv: https://arxiv.org/abs/2603.18184 · HTML: https://mlsanity.com/paper/2603.18184/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 53 references that are in the map.
- First posted on arXiv 18 Mar 2026.

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2301.02111/) (2023)
- [Language-agnostic BERT Sentence Embedding](https://mlsanity.com/paper/2007.01852/) (2020)
- [ByT5: Towards a token-free future with pre-trained byte-to-byte models](https://mlsanity.com/paper/2105.13626/) (2021)
- [CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation](https://mlsanity.com/paper/2103.06874/) (2021)
- [From Matching to Generation: A Survey on Generative Information Retrieval](https://mlsanity.com/paper/2404.14851/) (2024)
- [A Benchmark for Learning to Translate a New Language from One Grammar Book](https://mlsanity.com/paper/2309.16575/) (2023)
- [Composition-contrastive Learning for Sentence Embeddings](https://mlsanity.com/paper/2307.07380/) (2023)
- [Can we teach language models to gloss endangered languages?](https://mlsanity.com/paper/2406.18895/) (2024)
- [Hire a Linguist!: Learning Endangered Languages with In-Context Linguistic Descriptions](https://mlsanity.com/paper/2402.18025/) (2024)
- [GlossLM: A Massively Multilingual Corpus and Pretrained Model for Interlinear Glossed Text](https://mlsanity.com/paper/2403.06399/) (2024)
- [Tackling the Low-resource Challenge for Canonical Segmentation](https://mlsanity.com/paper/2010.02804/) (2020)
- [Wav2Gloss: Generating Interlinear Glossed Text from Speech](https://mlsanity.com/paper/2403.13169/) (2024)
- [A Simple Contrastive Framework Of Item Tokenization For Generative Recommendation](https://mlsanity.com/paper/2506.16683/) (2025)
- [Interdisciplinary Research in Conversation: A Case Study in Computational Morphology for Language Documentation](https://mlsanity.com/paper/2509.10644/) (2025)
- [SIGMORPHON 2023 Shared Task of Interlinear Glossing: Baseline Model](https://mlsanity.com/paper/2303.14234/) (2023)
- [How do Transformer Embeddings Represent Compositions? A Functional Analysis](https://mlsanity.com/paper/2506.00914/) (2025)
- [Massively Multilingual Joint Segmentation and Glossing](https://mlsanity.com/paper/2601.10925/) (2026)

## Abstract (arXiv)

Interlinear glossed text (IGT) is a standard notation for language documentation which is linguistically rich but laborious to produce manually. Recent automated IGT methods treat glosses as character sequences, neglecting their compositional structure. We propose CWoMP (Contrastive Word-Morpheme Pretraining), which instead treats morphemes as atomic form-meaning units with learned representations. A contrastively trained encoder aligns words-in-context with their constituent morphemes in a shared embedding space; an autoregressive decoder then generates the morpheme sequence by retrieving entries from a mutable lexicon of these embeddings. Predictions are interpretable--grounded in lexicon entries--and users can improve results at inference time by expanding the lexicon without retraining. We evaluate on diverse low-resource languages, showing that CWoMP outperforms existing methods while being significantly more efficient, with particularly strong gains in extremely low-resource settings.