# Neural Machine Translation Quality and Post-Editing Performance

arXiv: https://arxiv.org/abs/2109.05016 · HTML: https://mlsanity.com/paper/2109.05016/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 24 references that are in the map.
- First posted on arXiv 10 Sep 2021.

## Builds on (most cited first, 3 of 3)

- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Six Challenges for Neural Machine Translation](https://mlsanity.com/paper/1706.03872/) (2017)
- [Post-editing Productivity with Neural Machine Translation: An Empirical Assessment of Speed and Quality in the Banking and Finance Domain](https://mlsanity.com/paper/1906.01685/) (2019)

## Built on by (most cited first, 12 of 12)

- [Leveraging GPT-4 for Automatic Translation Post-Editing](https://mlsanity.com/paper/2305.14878/) (2023)
- [AI-Assisted Human Evaluation of Machine Translation](https://mlsanity.com/paper/2406.12419/) (2024)
- [Measuring Uncertainty in Translation Quality Evaluation (TQE)](https://mlsanity.com/paper/2111.07699/) (2021)
- [Extrinsic Evaluation of Machine Translation Metrics](https://mlsanity.com/paper/2212.10297/) (2022)
- [DivEMT: Neural Machine Translation Post-Editing Effort Across Typologically Diverse Languages](https://mlsanity.com/paper/2205.12215/) (2022)
- [WeTS: A Benchmark for Translation Suggestion](https://mlsanity.com/paper/2110.05151/) (2021)
- [Quality and Quantity of Machine Translation References for Automatic Metrics](https://mlsanity.com/paper/2401.01283/) (2024)
- [QE4PE: Word-level Quality Estimation for Human Post-Editing](https://mlsanity.com/paper/2503.03044/) (2025)
- [Beyond Human-Only: Evaluating Human-Machine Collaboration for Collecting High-Quality Translation Data](https://mlsanity.com/paper/2410.11056/) (2024)
- [Extending CREAMT: Leveraging Large Language Models for Literary Translation Post-Editing](https://mlsanity.com/paper/2504.03045/) (2025)
- [Findings of the WMT 2022 Shared Task on Translation Suggestion](https://mlsanity.com/paper/2211.16717/) (2022)
- [Hybrid Neural-LLM Pipeline for Morphological Glossing in Endangered Language Documentation: A Case Study of Jungar Tuvan](https://mlsanity.com/paper/2603.00923/) (2026)

## Abstract (arXiv)

We test the natural expectation that using MT in professional translation saves human processing time. The last such study was carried out by Sanchez-Torron and Koehn (2016) with phrase-based MT, artificially reducing the translation quality. In contrast, we focus on neural MT (NMT) of high quality, which has become the state-of-the-art approach since then and also got adopted by most translation companies. Through an experimental study involving over 30 professional translators for English -> Czech translation, we examine the relationship between NMT performance and post-editing time and quality. Across all models, we found that better MT systems indeed lead to fewer changes in the sentences in this industry setting. The relation between system quality and post-editing time is however not straightforward and, contrary to the results on phrase-based MT, BLEU is definitely not a stable predictor of the time or final output quality.