# A Komi-Yazva--Russian Parallel Corpus and Evaluation Protocol for Zero- and Few-Shot LLM Translation

arXiv: https://arxiv.org/abs/2606.06420 · HTML: https://mlsanity.com/paper/2606.06420/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 23 references that are in the map.
- First posted on arXiv 4 Jun 2026.

## Builds on (most cited first, 13 of 13)

- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation](https://mlsanity.com/paper/2302.09210/) (2023)
- [Dictionary-based Phrase-level Prompting of Large Language Models for Machine Translation](https://mlsanity.com/paper/2302.07856/) (2023)
- [Steering Large Language Models for Machine Translation with Finetuning and In-Context Learning](https://mlsanity.com/paper/2310.13448/) (2023)
- [Teaching Large Language Models an Unseen Language on the Fly](https://mlsanity.com/paper/2402.19167/) (2024)
- [In-Context Example Selection via Similarity Search Improves Low-Resource Machine Translation](https://mlsanity.com/paper/2408.00397/) (2024)
- [Machine Translation into Low-resource Language Varieties](https://mlsanity.com/paper/2106.06797/) (2021)
- [Understanding In-Context Machine Translation for Low-Resource Languages: A Case Study on Manchu](https://mlsanity.com/paper/2502.11862/) (2025)
- [Low-Resource Machine Translation through Retrieval-Augmented LLM Prompting: A Study on the Mambai Language](https://mlsanity.com/paper/2404.04809/) (2024)
- [Learning-From-Mistakes Prompting for Indigenous Language Translation](https://mlsanity.com/paper/2407.13343/) (2024)
- [Compensating for Data with Reasoning: Low-Resource Machine Translation with LLMs](https://mlsanity.com/paper/2505.22293/) (2025)
- [Evaluating OpenAI GPT Models for Translation of Endangered Uralic Languages: A Comparison of Reasoning and Non-Reasoning Architectures](https://mlsanity.com/paper/2512.16287/) (2025)
- [Towards Neural No-Resource Language Translation: A Comparative Evaluation of Approaches](https://mlsanity.com/paper/2412.20584/) (2024)

## Abstract (arXiv)

We present the first Komi-Yazva--Russian parallel corpus together with an explicit evaluation protocol for studying LLM translation in an endangered, extremely low-resource setting. The dataset contains 457 aligned sentence pairs from 74 narrative texts and is accompanied by documented provenance, sentence-level alignment, and story identifiers that enable leakage-aware evaluation. We use this setup to compare modern large language models on Komi-Yazva-to-Russian translation under severe parallel-data scarcity in zero-shot and retrieval-based few-shot regimes. The protocol includes story-level cross-validation, deterministic retrieval for few-shot prompting, strict validation of generated outputs, complementary reference-based and judge-based metrics, and story-level uncertainty estimates. Across models, LLMs produce non-trivial translations, but performance varies strongly by model family and prompting regime. Retrieval-based few-shot prompting consistently improves over zero-shot prompting, while gains beyond a small retrieved context remain limited. The results show that evaluative conclusions in this setting depend materially on metric choice and failure handling, so the paper frames the corpus as both a dataset contribution and a reproducible evaluation testbed for endangered-language machine translation.