mlsanityData to 7 Oct 2026

Data-Efficient Domain Adaptation for LLM-based MT using Contrastive Preference Optimization

arXiv 2510.27556 ↗First posted 31 Oct 2025cs.CLcited by 0

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

LLMs often require adaptation to domain-specific requirements, a process that can be expensive when relying solely on SFT. We present an empirical study on applying CPO to simulate a post-editing workflow for data-efficient domain adaptation. Our approach synthesizes preference pairs by treating the base model's own raw output as the 'rejected' translation and the human-approved TM entry as the 'chosen' one. This method provides direct feedback on the model's current knowledge, guiding it to align with domain-specific standards. Experiments in English-Brazilian Portuguese and English-Korean show that, by using just 14.7k preference pairs, the model achieves performance close to that of a model trained on 160k+ samples with SFT, demonstrating significant data efficiency. Although we showcase its effectiveness in MT, this application of CPO naturally generalizes to other generative tasks where a model's initial drafts can serve as a contrastive signal against a golden reference.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 14 of its 28 references that are in the map. First posted on arXiv 31 Oct 2025.

Builds on

14 of 28 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Advancing Translation Preference Modeling with RLHF: A Step Towards Cost-Effective Solution18 Feb
  2. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation16 Jan

Other references · 12

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2017Deep reinforcement learning from human preferencescited by 3,846
  6. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  7. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
  8. 2018A Survey of Domain Adaptation for Neural Machine Translationcited by 64
  9. 2023Steering Large Language Models for Machine Translation with Finetuning and In-Context Learningcited by 48
  10. 2024Fine-tuning Large Language Models for Domain-specific Machine Translationcited by 43
  11. 2021HOPE: A Task-Oriented and Human-Centric Evaluation Framework Using Professional Post-Editing Towards More Effective MT Evaluationcited by 17
  12. 2017The University of Edinburgh's Neural MT Systems for WMT17cited by 16

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024PANDA: Preference Adaptation for Enhancing Domain-Specific Abilities of LLMs
  2. 2024Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilities
  3. 2022Soft Alignment Objectives for Robust Adaptation of Language Generation
  4. 2026From Utterance to Vividity: Training Expressive Subtitle Translation LLM via Adaptive Local Preference Optimization
  5. 2026Domain-Specific Data Synthesis for LLMs via Minimal Sufficient Representation Learning
  6. 2024On Domain-Adaptive Post-Training for Multimodal Large Language Models
  7. 2024Is Preference Alignment Always the Best Option to Enhance LLM-Based Translation? An Empirical Analysis
  8. 2024Knowledge Editing in Language Models via Adapted Direct Preference Optimization