# Weakly Supervised Grammatical Error Correction using Iterative Decoding

arXiv: https://arxiv.org/abs/1811.01710 · HTML: https://mlsanity.com/paper/1811.01710/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2018.

## Built on by (most cited first, 3 of 3)

- [Character Transformations for Non-Autoregressive GEC Tagging](https://mlsanity.com/paper/2111.09280/) (2021)
- [A Syntax-Guided Grammatical Error Correction Model with Dependency Tree Correction](https://mlsanity.com/paper/2111.03294/) (2021)
- [Beyond Hard Samples: Robust and Effective Grammatical Error Correction with Cycle Self-Augmenting](https://mlsanity.com/paper/2310.13321/) (2023)

## Abstract (arXiv)

We describe an approach to Grammatical Error Correction (GEC) that is effective at making use of models trained on large amounts of weakly supervised bitext. We train the Transformer sequence-to-sequence model on 4B tokens of Wikipedia revisions and employ an iterative decoding strategy that is tailored to the loosely-supervised nature of the Wikipedia training corpus. Finetuning on the Lang-8 corpus and ensembling yields an F0.5 of 58.3 on the CoNLL'14 benchmark and a GLEU of 62.4 on JFLEG. The combination of weakly supervised training and iterative decoding obtains an F0.5 of 48.2 on CoNLL'14 even without using any labeled GEC data.