# The Unreasonable Effectiveness of Transformer Language Models in Grammatical Error Correction

arXiv: https://arxiv.org/abs/1906.01733 · HTML: https://mlsanity.com/paper/1906.01733/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jun 2019.

## Built on by (most cited first, 7 of 7)

- [Grammatical Error Correction: A Survey of the State of the Art](https://mlsanity.com/paper/2211.05166/) (2022)
- [Exploring Effectiveness of GPT-3 in Grammatical Error Correction: A Study on Performance and Controllability in Prompt-Based Methods](https://mlsanity.com/paper/2305.18156/) (2023)
- [Toward autocorrection of chemical process flowsheets using large language models](https://mlsanity.com/paper/2312.02873/) (2023)
- [Towards Effective Paraphrasing for Information Disguise](https://mlsanity.com/paper/2311.05018/) (2023)
- [Automatic Document Sketching: Generating Drafts from Analogous Texts](https://mlsanity.com/paper/2106.07192/) (2021)
- [Towards Lithuanian grammatical error correction](https://mlsanity.com/paper/2203.09963/) (2022)
- [A Language Model for Grammatical Error Correction in L2 Russian](https://mlsanity.com/paper/2307.01609/) (2023)

## Abstract (arXiv)

Recent work on Grammatical Error Correction (GEC) has highlighted the importance of language modeling in that it is certainly possible to achieve good performance by comparing the probabilities of the proposed edits. At the same time, advancements in language modeling have managed to generate linguistic output, which is almost indistinguishable from that of human-generated text. In this paper, we up the ante by exploring the potential of more sophisticated language models in GEC and offer some key insights on their strengths and weaknesses. We show that, in line with recent results in other NLP tasks, Transformer architectures achieve consistently high performance and provide a competitive baseline for future machine learning models.