# Multiple Sources are Better Than One: Incorporating External Knowledge in Low-Resource Glossing

arXiv: https://arxiv.org/abs/2406.11085 · HTML: https://mlsanity.com/paper/2406.11085/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 50 references that are in the map.
- First posted on arXiv 16 Jun 2024.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [ByT5: Towards a token-free future with pre-trained byte-to-byte models](https://mlsanity.com/paper/2105.13626/) (2021)
- [A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT](https://mlsanity.com/paper/2302.09419/) (2023)
- [SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings](https://mlsanity.com/paper/2004.08728/) (2020)
- [BanglaBERT: Language Model Pretraining and Benchmarks for Low-Resource Language Understanding Evaluation in Bangla](https://mlsanity.com/paper/2101.00204/) (2021)
- [Structured Attention Networks](https://mlsanity.com/paper/1702.00887/) (2017)
- [GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge](https://mlsanity.com/paper/1908.07245/) (2019)
- [Active Learning Principles for In-Context Learning with Large Language Models](https://mlsanity.com/paper/2305.14264/) (2023)
- [Multi-Source Neural Translation](https://mlsanity.com/paper/1601.00710/) (2016)
- [GlossLM: A Massively Multilingual Corpus and Pretrained Model for Interlinear Glossed Text](https://mlsanity.com/paper/2403.06399/) (2024)
- [Leveraging translations for speech transcription in low-resource settings](https://mlsanity.com/paper/1803.08991/) (2018)
- [CMULAB: An Open-Source Framework for Training and Deployment of Natural Language Processing Models](https://mlsanity.com/paper/2404.02408/) (2024)

## Built on by (most cited first, 5 of 5)

- [LingGym: How Far Are LLMs from Thinking Like Field Linguists?](https://mlsanity.com/paper/2511.00343/) (2025)
- [Interdisciplinary Research in Conversation: A Case Study in Computational Morphology for Language Documentation](https://mlsanity.com/paper/2509.10644/) (2025)
- [Prompt and circumstance: A word-by-word LLM prompting approach to interlinear glossing for low-resource languages](https://mlsanity.com/paper/2502.09778/) (2025)
- [Massively Multilingual Joint Segmentation and Glossing](https://mlsanity.com/paper/2601.10925/) (2026)
- [An Exploration-Analysis-Disambiguation Reasoning Framework for Word Sense Disambiguation with Low-Parameter LLMs](https://mlsanity.com/paper/2603.05400/) (2026)

## Abstract (arXiv)

In this paper, we address the data scarcity problem in automatic data-driven glossing for low-resource languages by coordinating multiple sources of linguistic expertise. We supplement models with translations at both the token and sentence level as well as leverage the extensive linguistic capability of modern LLMs. Our enhancements lead to an average absolute improvement of 5%-points in word-level accuracy over the previous state of the art on a typologically diverse dataset spanning six low-resource languages. The improvements are particularly noticeable for the lowest-resourced language Gitksan, where we achieve a 10%-point improvement. Furthermore, in a simulated ultra-low resource setting for the same six languages, training on fewer than 100 glossed sentences, we establish an average 10%-point improvement in word-level accuracy over the previous state-of-the-art system.