# LocQE: Principled Domain Adaptation for Localisation Quality Estimation by Leveraging Post-Edits

arXiv: https://arxiv.org/abs/2609.18720 · HTML: https://mlsanity.com/paper/2609.18720/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 41 references that are in the map.
- First posted on arXiv 16 Sep 2026.

## Builds on (most cited first, 27 of 27)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [Large Language Models Are State-of-the-Art Evaluators of Translation Quality](https://mlsanity.com/paper/2302.14520/) (2023)
- [Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation](https://mlsanity.com/paper/2104.14478/) (2021)
- [xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detection](https://mlsanity.com/paper/2310.10482/) (2023)
- [InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training](https://mlsanity.com/paper/2007.07834/) (2020)
- [CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Task](https://mlsanity.com/paper/2209.06243/) (2022)
- [Tokenization Is More Than Compression](https://mlsanity.com/paper/2402.18376/) (2024)
- [Error Span Annotation: A Balanced Approach for Human Evaluation of Machine Translation](https://mlsanity.com/paper/2406.11580/) (2024)
- [English Contrastive Learning Can Learn Universal Cross-lingual Sentence Embeddings](https://mlsanity.com/paper/2211.06127/) (2022)
- [BOUQuET: dataset, Benchmark and Open initiative for Universal Quality Evaluation in Translation](https://mlsanity.com/paper/2502.04314/) (2025)
- [Fine-Tuned Machine Translation Metrics Struggle in Unseen Domains](https://mlsanity.com/paper/2402.18747/) (2024)
- [Machine Translation Meta Evaluation through Translation Accuracy Challenge Sets](https://mlsanity.com/paper/2401.16313/) (2024)
- [Poor Man's Quality Estimation: Predicting Reference-Based MT Metrics Without the Reference](https://mlsanity.com/paper/2301.09008/) (2023)
- [Remedy: Learning Machine Translation Evaluation from Human Preferences with Reward Modeling](https://mlsanity.com/paper/2504.13630/) (2025)
- [Cross-lingual Human-Preference Alignment for Neural Machine Translation with Direct Quality Optimization](https://mlsanity.com/paper/2409.17673/) (2024)
- [A Parallel Evaluation Data Set of Software Documentation with Document Structure Annotation](https://mlsanity.com/paper/2008.04550/) (2020)
- [Tailoring Domain Adaptation for Machine Translation Quality Estimation](https://mlsanity.com/paper/2304.08891/) (2023)
- [ACES: Translation Accuracy Challenge Sets at WMT 2023](https://mlsanity.com/paper/2311.01153/) (2023)
- [Don't Sweat the Small Stuff: Segment-Level Meta-Evaluation Based on Pairwise Difference Correlation](https://mlsanity.com/paper/2509.25546/) (2025)
- [Who Watches the Watchmen? Humans Disagree With Translation Metrics on Unseen Domains](https://mlsanity.com/paper/2604.17393/) (2026)
- [PEAR: Pairwise Evaluation for Automatic Relative Scoring in Machine Translation](https://mlsanity.com/paper/2601.18006/) (2026)

## Abstract (arXiv)

Learned quality estimation (QE) models such as COMETKiwi are widespread and work well for general machine translation evaluation. However, they are known to struggle on unseen domains, limiting their performance in a real-world localisation context. We show that they are insensitive to some important factors in localisation, such as whether numbers are translated accurately, or even whether the correct number of spaces and punctuation are preserved in a translation. Further, a key capability for optimisation of machine translation is the ability of QE models to accurately rank different translations of a single segment, which suffers significantly from the domain transfer. In the absence of large-scale direct assessment data, we propose principled fine-tuning approaches to reduce the domain gap with even small amounts of post-editing data. Using a multi-task fine-tuning approach and a simple tokeniser intervention, we create a QE model which proves markedly better at distinguishing preferred post-edits from rejected initial translations in a localisation context. We show that preferences and artificial continuous scores stabilise each other, and argue that to calibrate metrics both in terms of their absolute scores and comparisons between translation of the same source, both types of signal are needed.