Setting up the Data Printer with Improved English to Ukrainian Machine Translation
Not on the map. Why?
Abstract (arXiv)
To build large language models for Ukrainian we need to expand our corpora with large amounts of new algorithmic tasks expressed in natural language. Examples of task performance expressed in English are abundant, so with a high-quality translation system our community will be enabled to curate datasets faster. To aid this goal, we introduce a recipe to build a translation system using supervised finetuning of a large pretrained language model with a noisy parallel dataset of 3M pairs of Ukrainian and English sentences followed by a second phase of training using 17K examples selected by k-fold perplexity filtering on another dataset of higher quality. Our decoder-only model named Dragoman beats performance of previous state of the art encoder-decoder models on the FLORES devtest set.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 17 of its 34 references that are in the map. First posted on arXiv 23 Apr 2024.
Builds on
17 of 34 references.
References · 17
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
- 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
- 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
- 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
- 2022Transformers learn in-context by gradient descentcited by 714
- 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
- 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
- 2020The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MTcited by 127
- 2018Correcting Length Bias in Neural Machine Translationcited by 119
- 2021The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformerscited by 117
5 more
- 2024How to Train Data-Efficient LLMscited by 91
- 2021WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language modelscited by 89
- 2021Unsupervised Neural Machine Translation with Generative Language Models Onlycited by 24
- 2021UA-GEC: Grammatical Error Correction and Fluency Corpus for the Ukrainian Languagecited by 19
- 2022BootAug: Boosting Text Augmentation via Hybrid Instance Filtering Frameworkcited by 1
Built on by
Citing papers per year: 2024: 2.
2 papers.