# Levenshtein Training for Word-level Quality Estimation

arXiv: https://arxiv.org/abs/2109.05611 · HTML: https://mlsanity.com/paper/2109.05611/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 20 references that are in the map.
- First posted on arXiv 12 Sep 2021.

## Builds on (most cited first, 9 of 9)

- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [Levenshtein Transformer](https://mlsanity.com/paper/1905.11006/) (2019)
- [Facebook FAIR's WMT19 News Translation Task Submission](https://mlsanity.com/paper/1907.06616/) (2019)
- [Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasing](https://mlsanity.com/paper/2004.14564/) (2020)
- [OpenKiwi: An Open Source Framework for Quality Estimation](https://mlsanity.com/paper/1902.08646/) (2019)
- [Paraphrase Generation as Zero-Shot Multilingual Translation: Disentangling Semantic Similarity from Lexical and Syntactic Diversity](https://mlsanity.com/paper/2008.04935/) (2020)
- [TransQuest at WMT2020: Sentence-Level Direct Assessment](https://mlsanity.com/paper/2010.05318/) (2020)

## Built on by (most cited first, 3 of 3)

- [SALMAN: Stability Analysis of Language Models Through the Maps Between Graph-based Manifolds](https://mlsanity.com/paper/2508.18306/) (2025)
- [The JHU-Microsoft Submission for WMT21 Quality Estimation Shared Task](https://mlsanity.com/paper/2109.08724/) (2021)
- [QUAK: A Synthetic Quality Estimation Dataset for Korean-English Neural Machine Translation](https://mlsanity.com/paper/2209.15285/) (2022)

## Abstract (arXiv)

We propose a novel scheme to use the Levenshtein Transformer to perform the task of word-level quality estimation. A Levenshtein Transformer is a natural fit for this task: trained to perform decoding in an iterative manner, a Levenshtein Transformer can learn to post-edit without explicit supervision. To further minimize the mismatch between the translation task and the word-level QE task, we propose a two-stage transfer learning procedure on both augmented data and human post-editing data. We also propose heuristics to construct reference labels that are compatible with subword-level finetuning and inference. Results on WMT 2020 QE shared task dataset show that our proposed method has superior data efficiency under the data-constrained setting and competitive performance under the unconstrained setting.