# M2PO: Multi-Perspective Multi-Pair Preference Optimization for Machine Translation

arXiv: https://arxiv.org/abs/2510.13434 · HTML: https://mlsanity.com/paper/2510.13434/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 42 of its 60 references that are in the map.
- First posted on arXiv 15 Oct 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [LLM preference optimization methods](https://mlsanity.com/alignment/llm-preference-optimization-methods/)

## Builds on (most cited first, 30 of 42)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [SimPO: Simple Preference Optimization with a Reference-Free Reward](https://mlsanity.com/paper/2405.14734/) (2024)
- [ORPO: Monolithic Preference Optimization without Reference Model](https://mlsanity.com/paper/2403.07691/) (2024)
- [Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation](https://mlsanity.com/paper/2401.08417/) (2024)
- [xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detection](https://mlsanity.com/paper/2310.10482/) (2023)
- [KTO: Model Alignment as Prospect Theoretic Optimization](https://mlsanity.com/paper/2402.01306/) (2024)
- [Tower: An Open Multilingual Large Language Model for Translation-Related Tasks](https://mlsanity.com/paper/2402.17733/) (2024)
- [Hallucinations in Large Multilingual Translation Models](https://mlsanity.com/paper/2303.16104/) (2023)
- [LiPO: Listwise Preference Optimization through Learning-to-Rank](https://mlsanity.com/paper/2402.01878/) (2024)
- [Scaling up COMETKIWI: Unbabel-IST 2023 Submission for the Quality Estimation Shared Task](https://mlsanity.com/paper/2309.11925/) (2023)
- [Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Better](https://mlsanity.com/paper/2212.08597/) (2022)
- [The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harm](https://mlsanity.com/paper/2406.18682/) (2024)
- [On the Limitations of Reference-Free Evaluations of Generated Text](https://mlsanity.com/paper/2210.12563/) (2022)
- [TIM: Teaching Large Language Models to Translate with Comparison](https://mlsanity.com/paper/2307.04408/) (2023)
- [LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languages](https://mlsanity.com/paper/2407.05975/) (2024)
- [MT-R1-Zero: Advancing LLM-based Machine Translation via R1-Zero-like Reinforcement Learning](https://mlsanity.com/paper/2504.10160/) (2025)
- [Tower+: Bridging Generality and Translation Specialization in Multilingual LLMs](https://mlsanity.com/paper/2506.17080/) (2025)
- [Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Study](https://mlsanity.com/paper/2502.02481/) (2025)
- [X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scale](https://mlsanity.com/paper/2410.03115/) (2024)
- [HalOmi: A Manually Annotated Benchmark for Multilingual Hallucination and Omission Detection in Machine Translation](https://mlsanity.com/paper/2305.11746/) (2023)
- [Improving Machine Translation with Human Feedback: An Exploration of Quality Estimation as a Reward Model](https://mlsanity.com/paper/2401.12873/) (2024)
- [Language Imbalance Driven Rewarding for Multilingual Self-improving](https://mlsanity.com/paper/2410.08964/) (2024)
- [Word Alignment as Preference for Machine Translation](https://mlsanity.com/paper/2405.09223/) (2024)
- [As Little as Possible, as Much as Necessary: Detecting Over- and Undertranslations with Contrastive Conditioning](https://mlsanity.com/paper/2203.01927/) (2022)
- [Fine-Grained Reward Optimization for Machine Translation using Error Severity Mappings](https://mlsanity.com/paper/2411.05986/) (2024)

## Built on by (most cited first, 4 of 4)

- [Cross-Preference Learning for Sentence-Level and Context-Aware Machine Translation](https://mlsanity.com/paper/2603.25183/) (2026)
- [STAR : Sentence Translation Alignment Rate for Document-to-Document Machine Translation](https://mlsanity.com/paper/2608.27161/) (2026)
- [RIMS: Preference Optimization via Smoothed Multi-pair Aggregation for Small-Scale LLM Retrieval-Augmented Generation](https://mlsanity.com/paper/2607.16431/) (2026)
- [Qwen-MT-Lumen at WMT2026: Instruction-Specialized Translation via Quality-Aware Training and On-Policy Distillation](https://mlsanity.com/paper/2610.19758/) (2026)

## Abstract (arXiv)

Aligning Large Language Models (LLMs) with human preferences is pivotal for Machine Translation (MT), yet current approaches are often hindered by misleading reward signals. Our analysis reveals that prevailing Quality Estimation (QE) models exhibit a systematic blind spot toward partial errors, specifically partial hallucinations and omissions, often favoring superficially fluent but unfaithful translations. To address this issue, we propose M2PO (Multi-Perspective Multi-Pair Preference Optimization), a data-centric framework for preference optimization in machine translation. First, to correct the bias toward fluency, M2PO uses a dual-perspective mechanism that decouples semantic fidelity from fluency and prioritizes faithfulness through a curriculum strategy. Second, after correcting this bias, partial errors fall between perfect and severely incorrect translations, making them difficult to learn through standard best-versus-worst comparisons. We therefore introduce a multi-pair objective that leverages the full candidate list to capture these fine-grained error signals. Experiments on WMT23, WMT24, and FLORES-200 show that M2PO enables a 9B model to outperform leading open-source baselines and achieve parity with proprietary models such as GPT-4o and Gemini-2.0-Flash, demonstrating strong potential for efficient and high-fidelity LLM-based translation. Our code and dataset will be released.