mlsanityData to 14 Oct 2026

Direct Preference Optimization for Neural Machine Translation with Minimum Bayes Risk Decoding

arXiv 2311.08380 ↗First posted 14 Nov 2023cs.CLcited by 28

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Minimum Bayes Risk (MBR) decoding can significantly improve translation performance of Multilingual Large Language Models (MLLMs). However, MBR decoding is computationally expensive. We show how the recently developed Reinforcement Learning technique, Direct Preference Optimization (DPO), can fine-tune MLLMs to get the gains of MBR without any additional computation in inference. Our method uses only a small monolingual fine-tuning set and yields significantly improved performance on multiple NMT test sets compared to MLLMs without DPO.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 16 of its 29 references that are in the map. First posted on arXiv 14 Nov 2023.

Builds on

16 of 29 references.

References · 16

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  2. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  3. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  4. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  5. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 843
  6. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 686
  7. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 500
  8. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  9. 2020The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MTcited by 128
  10. 2021High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural Metricscited by 104
  11. 2023The unreasonable effectiveness of few-shot learning for machine translationcited by 91
  12. 2021Sampling-Based Approximations to Minimum Bayes Risk Decoding for Neural Machine Translationcited by 65
4 more
  1. 2023Epsilon Sampling Rocks: Investigating Sampling Strategies for Minimum Bayes Risk Decoding for Machine Translationcited by 58
  2. 2022Follow the Wisdom of the Crowd: Effective Text Generation via Minimum Bayes Risk Decodingcited by 54
  3. 2023TIM: Teaching Large Language Models to Translate with Comparisoncited by 50
  4. 2023MBR and QE Finetuning: Training-time Distillation of the Best and Most Expensive Decoding Methodscited by 33

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 15; 2025: 7; 2026: 6.

28 papers.

Sort:

By area:

  1. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 51
  2. 2024Online DPO: Online Direct Preference Optimization with Fast-Slow Chasingcited by 39
  3. 2024Regularized Best-of-N Sampling with Minimum Bayes Risk Objective for Language Model Alignmentcited by 25
  4. 2024Linear-time Minimum Bayes Risk Decoding with Reference Aggregationcited by 21
  5. 2024Better Instruction-Following Through Minimum Bayes Riskcited by 19
  6. 2024Can Automatic Metrics Assess High-Quality Translations?cited by 16
  7. 2024Mitigating Metric Bias in Minimum Bayes Risk Decodingcited by 16
  8. 2024Hyperparameter-Free Approach for Faster Minimum Bayes Risk Decodingcited by 15
  9. 2026CycleVLA: Proactive Self-Correcting Vision-Language-Action Models via Subtask Backtracking and Minimum Bayes Risk Decodingcited by 14
  10. 2025Uncertainty-Aware Decoding with Minimum Bayes Riskcited by 11
  11. 2024On Extending Direct Preference Optimization to Accommodate Tiescited by 8
  12. 2024Modeling User Preferences with Automatic Metrics: Creating a High-Quality Preference Dataset for Machine Translationcited by 8
16 more
  1. 2024Cross-lingual Human-Preference Alignment for Neural Machine Translation with Direct Quality Optimizationcited by 5
  2. 2024Is Preference Alignment Always the Best Option to Enhance LLM-Based Translation? An Empirical Analysiscited by 4
  3. 2025Advancing Decoding Strategies: Enhancements in Locally Typical Sampling for LLMscited by 4
  4. 2024Annotation-Efficient Language Model Alignment via Diverse and Representative Response Textscited by 3
  5. 2025CRPO: Confidence-Reward Driven Preference Optimization for Machine Translationcited by 3
  6. 2024On the True Distribution Approximation of Minimum Bayes-Risk Decodingcited by 2
  7. 2025Calibrating Translation Decoding with Quality Estimation on LLMscited by 2
  8. 2025Structure-Conditional Minimum Bayes Risk Decodingcited by 2
  9. 2026Cross-Preference Learning for Sentence-Level and Context-Aware Machine Translationcited by 2
  10. 2025Minimum Bayes Risk Decoding for Error Span Detection in Reference-Free Automatic Machine Translation Evaluationcited by 1
  11. 2026Consensus Group Relative Policy Optimization for Text Generationcited by 1
  12. 2026STAR : Sentence Translation Alignment Rate for Document-to-Document Machine Translationcited by 1
  13. 2024PMMT: Preference Alignment in Multilingual Machine Translation via LLM Distillationcited by 0
  14. 2025Document-Level Text Generation with Minimum Bayes Risk Decoding using Optimal Transportcited by 0
  15. 2026BERAG: Bayesian Ensemble Retrieval-Augmented Generation for Knowledge-based Visual Question Answeringcited by 0
  16. 2026Enhancing Small Language Models for Power Outage Report Generation via Minimum Risk Trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Backtranslation Augmented Direct Preference Optimization for Neural Machine Translation
  2. 2024Multi-Reference Preference Optimization for Large Language Models
  3. 2024Intermediate direct preference optimization
  4. 2025Lightweight Robust Direct Preference Optimization
  5. 2024Active Preference Learning for Large Language Models
  6. 2024Enhancing LLM Safety via Constrained Direct Preference Optimization
  7. 2024Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive
  8. 2024Optimizing LLMs with Direct Preferences: A Data Efficiency Perspective