# Can LLMs Really Learn to Translate a Low-Resource Language from One Grammar Book?

arXiv: https://arxiv.org/abs/2409.19151 · HTML: https://mlsanity.com/paper/2409.19151/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 71 references that are in the map.
- First posted on arXiv 27 Sep 2024.

## Builds on (most cited first, 30 of 32)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [RULER: What's the Real Context Size of Your Long-Context Language Models?](https://mlsanity.com/paper/2404.06654/) (2024)
- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [ByT5: Towards a token-free future with pre-trained byte-to-byte models](https://mlsanity.com/paper/2105.13626/) (2021)
- [A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models](https://mlsanity.com/paper/2309.11674/) (2023)
- [A Benchmark for Learning to Translate a New Language from One Grammar Book](https://mlsanity.com/paper/2309.16575/) (2023)
- [Dictionary-based Phrase-level Prompting of Large Language Models for Machine Translation](https://mlsanity.com/paper/2302.07856/) (2023)
- [LLMs Are Few-Shot In-Context Low-Resource Language Learners](https://mlsanity.com/paper/2403.16512/) (2024)
- [Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing](https://mlsanity.com/paper/1807.00914/) (2018)
- [Some Languages are More Equal than Others: Probing Deeper into the Linguistic Disparity in the NLP World](https://mlsanity.com/paper/2210.08523/) (2022)
- [Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale](https://mlsanity.com/paper/2203.00633/) (2022)
- [Learning Language Representations for Typology Prediction](https://mlsanity.com/paper/1707.09569/) (2017)
- [Navigating the Metrics Maze: Reconciling Score Magnitudes and Accuracies](https://mlsanity.com/paper/2401.06760/) (2024)
- [Teaching Large Language Models an Unseen Language on the Fly](https://mlsanity.com/paper/2402.19167/) (2024)
- [Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations](https://mlsanity.com/paper/2004.14923/) (2020)
- [Continuous multilinguality with language vectors](https://mlsanity.com/paper/1612.07486/) (2016)
- [Shortcomings of LLMs for Low-Resource Translation: Retrieval and Understanding are Both the Problem](https://mlsanity.com/paper/2406.15625/) (2024)
- [Predicting Target Language CCG Supertags Improves Neural Machine Translation](https://mlsanity.com/paper/1702.01147/) (2017)
- [Are Character-level Translations Worth the Wait? Comparing ByT5 and mT5 for Machine Translation](https://mlsanity.com/paper/2302.14220/) (2023)
- [Can we teach language models to gloss endangered languages?](https://mlsanity.com/paper/2406.18895/) (2024)
- [Quality or Quantity? On Data Scale and Diversity in Adapting Large Language Models for Low-Resource Translation](https://mlsanity.com/paper/2408.12780/) (2024)
- [Hire a Linguist!: Learning Endangered Languages with In-Context Linguistic Descriptions](https://mlsanity.com/paper/2402.18025/) (2024)
- [Low-Resource Machine Translation through Retrieval-Augmented LLM Prompting: A Study on the Mambai Language](https://mlsanity.com/paper/2404.04809/) (2024)
- [Natural Language Processing RELIES on Linguistics](https://mlsanity.com/paper/2405.05966/) (2024)
- [LLM-Assisted Rule Based Machine Translation for Low/No-Resource Languages](https://mlsanity.com/paper/2405.08997/) (2024)

## Built on by (most cited first, 23 of 23)

- [Understanding In-Context Machine Translation for Low-Resource Languages: A Case Study on Manchu](https://mlsanity.com/paper/2502.11862/) (2025)
- [Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translation](https://mlsanity.com/paper/2504.01919/) (2025)
- [LingBench++: A Linguistically-Informed Benchmark and Reasoning Framework for Multi-Step and Cross-Cultural Inference with LLMs](https://mlsanity.com/paper/2507.16809/) (2025)
- [Read it in Two Steps: Translating Extremely Low-Resource Languages with Code-Augmented Grammar Books](https://mlsanity.com/paper/2506.01796/) (2025)
- [Compensating for Data with Reasoning: Low-Resource Machine Translation with LLMs](https://mlsanity.com/paper/2505.22293/) (2025)
- [Explain-then-Process: Using Grammar Prompting to Enhance Grammatical Acceptability Judgments](https://mlsanity.com/paper/2506.02302/) (2025)
- [Prompt and circumstance: A word-by-word LLM prompting approach to interlinear glossing for low-resource languages](https://mlsanity.com/paper/2502.09778/) (2025)
- [The Role of Mixed-Language Documents for Multilingual Large Language Model Pretraining](https://mlsanity.com/paper/2601.00364/) (2026)
- [Massively Multilingual Joint Segmentation and Glossing](https://mlsanity.com/paper/2601.10925/) (2026)
- [Speaking of Language: Reflections on Metalanguage Research in NLP](https://mlsanity.com/paper/2604.02645/) (2026)
- [Testing the Limits of Machine Translation from One Book](https://mlsanity.com/paper/2508.06665/) (2025)
- [Do Large Language Models Grasp The Grammar? Evidence from Grammar-Book-Guided Probing in Luxembourgish](https://mlsanity.com/paper/2510.24856/) (2025)
- [Rashid: A Cipher-Based Framework for Exploring In-Context Language Learning](https://mlsanity.com/paper/2603.22497/) (2026)
- [Open Machine Translation for Esperanto](https://mlsanity.com/paper/2603.29345/) (2026)
- [Evaluating In-Context Translation with Synchronous Context-Free Grammar Transduction](https://mlsanity.com/paper/2604.07320/) (2026)
- [Reinforcement Learning Elicits Contextual Learning of Unseen Language Translation](https://mlsanity.com/paper/2606.06428/) (2026)
- [LLMs in the Real World: Evaluating "AI" in Emergency Contexts](https://mlsanity.com/paper/2607.00019/) (2026)
- [A POS Tier Is the Key to Automated Annotation for Low-Resource Language Documentation: Neural Interlinear Glossing for Irabu, a Southern Ryukyuan Language](https://mlsanity.com/paper/2607.13372/) (2026)
- [A Factorial Study of Synthetic Data Generation for Low-Resource Machine Translation using Grammar Books](https://mlsanity.com/paper/2607.22376/) (2026)
- [The Interlingua Hypothesis: LLMs Translate via a Latent Task-agnostic Feature Space](https://mlsanity.com/paper/2609.00515/) (2026)
- [Typological Feature Prediction with Large Language Models: An In-Context Learning Approach](https://mlsanity.com/paper/2609.03775/) (2026)
- [LLM Agents as Computational Typologists](https://mlsanity.com/paper/2609.07791/) (2026)
- [Leveraging Fine-grained Error Correction in Korean Speech Recognition for Consultation Services](https://mlsanity.com/paper/2609.09889/) (2026)

## Abstract (arXiv)

Extremely low-resource (XLR) languages lack substantial corpora for training NLP models, motivating the use of all available resources such as dictionaries and grammar books. Machine Translation from One Book (Tanzer et al., 2024) suggests that prompting long-context LLMs with one grammar book enables English-Kalamang translation, an XLR language unseen by LLMs - a noteworthy case of linguistics helping an NLP task. We investigate the source of this translation ability, finding almost all improvements stem from the book's parallel examples rather than its grammatical explanations. We find similar results for Nepali and Guarani, seen low-resource languages, and we achieve performance comparable to an LLM with a grammar book by simply fine-tuning an encoder-decoder translation model. We then investigate where grammar books help by testing two linguistic tasks, grammaticality judgment and gloss prediction, and we explore what kind of grammatical knowledge helps by introducing a typological feature prompt that achieves leading results on these more relevant tasks. We thus emphasise the importance of task-appropriate data for XLR languages: parallel examples for translation, and grammatical data for linguistic tasks. As we find no evidence that long-context LLMs can make effective use of grammatical explanations for XLR translation, we conclude data collection for multilingual XLR tasks such as translation is best focused on parallel data over linguistic description.