mlsanityData to 7 Oct 2026

Multiple Sources are Better Than One: Incorporating External Knowledge in Low-Resource Glossing

arXiv 2406.11085 ↗First posted 16 Jun 2024cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

In this paper, we address the data scarcity problem in automatic data-driven glossing for low-resource languages by coordinating multiple sources of linguistic expertise. We supplement models with translations at both the token and sentence level as well as leverage the extensive linguistic capability of modern LLMs. Our enhancements lead to an average absolute improvement of 5%-points in word-level accuracy over the previous state of the art on a typologically diverse dataset spanning six low-resource languages. The improvements are particularly noticeable for the lowest-resourced language Gitksan, where we achieve a 10%-point improvement. Furthermore, in a simulated ultra-low resource setting for the same six languages, training on fewer than 100 glossed sentences, we establish an average 10%-point improvement in word-level accuracy over the previous state-of-the-art system.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 20 of its 50 references that are in the map. First posted on arXiv 16 Jun 2024.

Builds on

20 of 50 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  7. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  8. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  9. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  10. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 504
  11. 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 364
  12. 2020SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddingscited by 155
8 more
  1. 2021BanglaBERT: Language Model Pretraining and Benchmarks for Low-Resource Language Understanding Evaluation in Banglacited by 123
  2. 2017Structured Attention Networkscited by 95
  3. 2019GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledgecited by 78
  4. 2023Active Learning Principles for In-Context Learning with Large Language Modelscited by 53
  5. 2016Multi-Source Neural Translationcited by 37
  6. 2024GlossLM: A Massively Multilingual Corpus and Pretrained Model for Interlinear Glossed Textcited by 12
  7. 2018Leveraging translations for speech transcription in low-resource settingscited by 5
  8. 2024CMULAB: An Open-Source Framework for Training and Deployment of Natural Language Processing Modelscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025LingGym: How Far Are LLMs from Thinking Like Field Linguists?cited by 6
  2. 2025Interdisciplinary Research in Conversation: A Case Study in Computational Morphology for Language Documentationcited by 4
  3. 2025Prompt and circumstance: A word-by-word LLM prompting approach to interlinear glossing for low-resource languagescited by 1
  4. 2026Massively Multilingual Joint Segmentation and Glossingcited by 1
  5. 2026An Exploration-Analysis-Disambiguation Reasoning Framework for Word Sense Disambiguation with Low-Parameter LLMscited by 1