mlsanityData to 14 Oct 2026

Improving LLMs for Machine Translation Using Synthetic Preference Data

arXiv 2508.14951 ↗First posted 20 Aug 2025cs.CLcited by 2

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Large language models have emerged as effective machine translation systems. In this paper, we explore how a general instruction-tuned large language model can be improved for machine translation using relatively few easily produced data resources. Using Slovene as a use case, we improve the GaMS-9B-Instruct model using Direct Preference Optimization (DPO) training on a programmatically curated and enhanced subset of a public dataset. As DPO requires pairs of quality-ranked instances, we generated its training dataset by translating English Wikipedia articles using two LLMs, GaMS-9B-Instruct and EuroLLM-9B-Instruct. We ranked the resulting translations based on heuristics coupled with automatic evaluation metrics such as COMET. The evaluation shows that our fine-tuned model outperforms both models involved in the dataset generation. In comparison to the baseline models, the fine-tuned model achieved a COMET score gain of around 0.04 and 0.02, respectively, on translating Wikipedia articles. It also more consistently avoids language and formatting errors.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 14 of its 25 references that are in the map. First posted on arXiv 20 Aug 2025.

Builds on

14 of 25 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Curry-DPO: Enhancing Alignment using Curriculum Learning & Ranked Preferences12 Mar

Other references · 13

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2017Deep reinforcement learning from human preferencescited by 3,912
  4. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  5. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  6. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  7. 2016Bag of Tricks for Efficient Text Classificationcited by 876
  8. 2016Training Deep Nets with Sublinear Memory Costcited by 848
  9. 2016FastText.zip: Compressing text classification modelscited by 335
  10. 2022CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Taskcited by 207
  11. 2023ZeRO++: Extremely Efficient Collective Communication for Giant Model Trainingcited by 52
  12. 2024Curriculum Direct Preference Optimization for Diffusion and Consistency Modelscited by 36
1 more
  1. 2024Generative Model for Less-Resourced Language with 1 billion parameterscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Building a Strong Instruction Language Model for a Less-Resourced Languagecited by 1
  2. 2026Backtranslation Augmented Direct Preference Optimization for Neural Machine Translationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation
  2. 2025Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translation
  3. 2024A Preference-driven Paradigm for Enhanced Translation with Large Language Models
  4. 2024What do Large Language Models Need for Machine Translation Evaluation?
  5. 2024PMMT: Preference Alignment in Multilingual Machine Translation via LLM Distillation
  6. 2024A Survey on Data Synthesis and Augmentation for Large Language Models
  7. 2024Knowledge Editing in Language Models via Adapted Direct Preference Optimization
  8. 2024Modeling User Preferences with Automatic Metrics: Creating a High-Quality Preference Dataset for Machine Translation