# Neural versus Phrase-Based Machine Translation Quality: a Case Study

arXiv: https://arxiv.org/abs/1608.04631 · HTML: https://mlsanity.com/paper/1608.04631/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Aug 2016.

## Built on by (most cited first, 19 of 19)

- [ACES: Translation Accuracy Challenge Sets for Evaluating Machine Translation Metrics](https://mlsanity.com/paper/2210.15615/) (2022)
- [HOPE: A Task-Oriented and Human-Centric Evaluation Framework Using Professional Post-Editing Towards More Effective MT Evaluation](https://mlsanity.com/paper/2112.13833/) (2021)
- [MENLI: Robust Evaluation Metrics from Natural Language Inference](https://mlsanity.com/paper/2208.07316/) (2022)
- [This is not a Dataset: A Large Negation Benchmark to Challenge Large Language Models](https://mlsanity.com/paper/2310.15941/) (2023)
- [Geometric Signatures of Compositionality Across a Language Model's Lifetime](https://mlsanity.com/paper/2410.01444/) (2024)
- [DivEMT: Neural Machine Translation Post-Editing Effort Across Typologically Diverse Languages](https://mlsanity.com/paper/2205.12215/) (2022)
- [Translation in the Hands of Many:Centering Lay Users in Machine Translation Interactions](https://mlsanity.com/paper/2502.13780/) (2025)
- [Revisiting Negation in Neural Machine Translation](https://mlsanity.com/paper/2107.12203/) (2021)
- [Translating the Unseen? Yoruba-English MT in Low-Resource, Morphologically-Unmarked Settings](https://mlsanity.com/paper/2103.04225/) (2021)
- [The Ecological Footprint of Neural Machine Translation Systems](https://mlsanity.com/paper/2202.02170/) (2022)
- [Good, but not always Fair: An Evaluation of Gender Bias for three commercial Machine Translation Systems](https://mlsanity.com/paper/2306.05882/) (2023)
- [Modeling Target-Side Morphology in Neural Machine Translation: A Comparison of Strategies](https://mlsanity.com/paper/2203.13550/) (2022)
- [Dict-NMT: Bilingual Dictionary based NMT for Extremely Low Resource Languages](https://mlsanity.com/paper/2206.04439/) (2022)
- [Transformer-Based Low-Resource Language Translation: A Study on Standard Bengali to Sylheti](https://mlsanity.com/paper/2510.18898/) (2025)
- [An empirical analysis of phrase-based and neural machine translation](https://mlsanity.com/paper/2103.03108/) (2021)
- [Negation-Instance Based Evaluation of End-to-End Negation Resolution](https://mlsanity.com/paper/2109.10013/) (2021)
- [Advancing Bangla Machine Translation Through Informal Datasets](https://mlsanity.com/paper/2512.13487/) (2025)
- [Challenges in Translating Technical Lectures: Insights from the NPTEL](https://mlsanity.com/paper/2602.08698/) (2026)
- [Metaphors in Literary Post-Editing: Opening Pandora's Box?](https://mlsanity.com/paper/2605.21178/) (2026)

## Abstract (arXiv)

Within the field of Statistical Machine Translation (SMT), the neural approach (NMT) has recently emerged as the first technology able to challenge the long-standing dominance of phrase-based approaches (PBMT). In particular, at the IWSLT 2015 evaluation campaign, NMT outperformed well established state-of-the-art PBMT systems on English-German, a language pair known to be particularly hard because of morphology and syntactic differences. To understand in what respects NMT provides better translation quality than PBMT, we perform a detailed analysis of neural versus phrase-based SMT outputs, leveraging high quality post-edits performed by professional translators on the IWSLT data. For the first time, our analysis provides useful insights on what linguistic phenomena are best modeled by neural models -- such as the reordering of verbs -- while pointing out other aspects that remain to be improved.