mlsanityData to 14 Oct 2026

Word Alignment as Preference for Machine Translation

arXiv 2405.09223 ↗First posted 15 May 2024cs.CL cs.AIcited by 14

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

The problem of hallucination and omission, a long-standing problem in machine translation (MT), is more pronounced when a large language model (LLM) is used in MT because an LLM itself is susceptible to these phenomena. In this work, we mitigate the problem in an LLM-based MT model by guiding it to better word alignment. We first study the correlation between word alignment and the phenomena of hallucination and omission in MT. Then we propose to utilize word alignment as preference to optimize the LLM-based MT model. The preference data are constructed by selecting chosen and rejected translations from multiple MT tools. Subsequently, direct preference optimization is used to optimize the LLM-based model towards the preference signal. Given the absence of evaluators specifically designed for hallucination and omission in MT, we further propose selecting hard instances and utilizing GPT-4 to directly evaluate the performance of the models in mitigating these issues. We verify the rationality of these designed evaluation methods by experiments, followed by extensive results demonstrating the effectiveness of word alignment-based preference optimization to mitigate hallucination and omission. On the other hand, although it shows promise in mitigating hallucination and omission, the overall performance of MT in different language directions remains mixed, with slight increases in BLEU and decreases in COMET.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 36 of its 54 references that are in the map. First posted on arXiv 15 May 2024.

Builds on

36 of 54 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation16 Jan
  2. 2023Zephyr: Direct Distillation of LM Alignment25 Oct
  3. 2022Training language models to follow instructions with human feedback4 Mar

Other references · 33

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  7. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  8. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  10. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  11. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  12. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
21 more
  1. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  2. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 726
  3. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  4. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 348
  5. 2023ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPTcited by 250
  6. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 206
  7. 2021Word Alignment by Fine-tuning Embeddings on Parallel Corporacited by 186
  8. 2020SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddingscited by 156
  9. 2016Modeling Coverage for Neural Machine Translationcited by 110
  10. 2022Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Bettercited by 75
  11. 2021Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignmentcited by 41
  12. 2023HalOmi: A Manually Annotated Benchmark for Multilingual Hallucination and Omission Detection in Machine Translationcited by 35
  13. 2022PCL: Peer-Contrastive Learning with Diverse Augmentations for Unsupervised Sentence Embeddingscited by 29
  14. 2020A Supervised Word Alignment Method based on Cross-Language Span Prediction using Multilingual BERTcited by 27
  15. 2024Enhancing Cross-lingual Sentence Embedding for Low-resource Languages with Word Alignmentcited by 13
  16. 2022As Little as Possible, as Much as Necessary: Detecting Over- and Undertranslations with Contrastive Conditioningcited by 11
  17. 2023WSPAlign: Word Alignment Pre-training via Large-Scale Weakly Supervised Span Predictioncited by 10
  18. 2023Dual-Alignment Pre-training for Cross-lingual Sentence Embeddingcited by 9
  19. 2023Leveraging Multi-lingual Positive Instances in Contrastive Learning to Improve Sentence Embeddingcited by 9
  20. 2023Towards reporting bias in visual-language datasets: bimodal augmentation by decoupling object-attribute associationcited by 7
  21. 2023VECO 2.0: Cross-lingual Language Model Pre-training with Multi-granularity Contrastive Learningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 6; 2026: 5.

14 papers.

Sort:

By area:

  1. 2025A Survey of Direct Preference Optimizationcited by 36
  2. 2024Multilingual Large Language Models: A Systematic Surveycited by 18
  3. 2024Direct Quantized Training of Language Models with Stochastic Roundingcited by 6
  4. 2025Redefining Machine Translation on Social Network Services with Large Language Modelscited by 6
  5. 2025Two Intermediate Translations Are Better Than One: Fine-tuning LLMs for Document-level Translation Refinementcited by 5
  6. 2024Is Preference Alignment Always the Best Option to Enhance LLM-Based Translation? An Empirical Analysiscited by 4
  7. 2025M2PO: Multi-Perspective Multi-Pair Preference Optimization for Machine Translationcited by 4
  8. 2025CRPO: Confidence-Reward Driven Preference Optimization for Machine Translationcited by 3
  9. 2025Test-Time Alignment for Large Language Models via Textual Model Predictive Controlcited by 2
  10. 2026Benchmarking Machine Translation on Chinese Social Media Textscited by 2
  11. 2026Cross-Preference Learning for Sentence-Level and Context-Aware Machine Translationcited by 2
  12. 2026Scaling Unsupervised Word Alignment to Documents via Structural Constraintscited by 1
2 more
  1. 2026STAR : Sentence Translation Alignment Rate for Document-to-Document Machine Translationcited by 1
  2. 2026Enhancing Document-Level Machine Translation via Filtered Synthetic Corpora and Two-Stage LLM Adaptationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Understanding Alignment in Multimodal LLMs: A Comprehensive Study
  2. 2025Mitigating Hallucinated Translations in Large Language Models with Hallucination-focused Preference Optimization
  3. 2024Modeling User Preferences with Automatic Metrics: Creating a High-Quality Preference Dataset for Machine Translation
  4. 2024PMMT: Preference Alignment in Multilingual Machine Translation via LLM Distillation
  5. 2025Data Selection for LLM Alignment Using Fine-Grained Preferences
  6. 2025Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translation
  7. 2024Paraphrase-Aligned Machine Translation
  8. 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges