# An Analysis of Source-Side Grammatical Errors in NMT

arXiv: https://arxiv.org/abs/1905.10024 · HTML: https://mlsanity.com/paper/1905.10024/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 May 2019.

## Built on by (most cited first, 6 of 6)

- [RuCoLA: Russian Corpus of Linguistic Acceptability](https://mlsanity.com/paper/2210.12814/) (2022)
- [CODET: A Benchmark for Contrastive Dialectal Evaluation of Machine Translation](https://mlsanity.com/paper/2305.17267/) (2023)
- [Did Translation Models Get More Robust Without Anyone Even Noticing?](https://mlsanity.com/paper/2403.03923/) (2024)
- [Autocorrect in the Process of Translation -- Multi-task Learning Improves Dialogue Machine Translation](https://mlsanity.com/paper/2103.16189/) (2021)
- [Misspellings in Natural Language Processing: A survey](https://mlsanity.com/paper/2501.16836/) (2025)
- [A Representation Level Analysis of NMT Model Robustness to Grammatical Errors](https://mlsanity.com/paper/2505.21224/) (2025)

## Abstract (arXiv)

The quality of Neural Machine Translation (NMT) has been shown to significantly degrade when confronted with source-side noise. We present the first large-scale study of state-of-the-art English-to-German NMT on real grammatical noise, by evaluating on several Grammar Correction corpora. We present methods for evaluating NMT robustness without true references, and we use them for extensive analysis of the effects that different grammatical errors have on the NMT output. We also introduce a technique for visualizing the divergence distribution caused by a source-side error, which allows for additional insights.