# Stronger Baselines for Grammatical Error Correction Using Pretrained Encoder-Decoder Model

arXiv: https://arxiv.org/abs/2005.11849 · HTML: https://mlsanity.com/paper/2005.11849/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 May 2020.

## Built on by (most cited first, 30 of 33)

- [A Simple Recipe for Multilingual Grammatical Error Correction](https://mlsanity.com/paper/2106.03830/) (2021)
- [Is ChatGPT a Highly Fluent Grammatical Error Correction System? A Comprehensive Evaluation](https://mlsanity.com/paper/2304.01746/) (2023)
- [Grammatical Error Correction: A Survey of the State of the Art](https://mlsanity.com/paper/2211.05166/) (2022)
- [MuCGEC: a Multi-Reference Multi-Source Evaluation Dataset for Chinese Grammatical Error Correction](https://mlsanity.com/paper/2204.10994/) (2022)
- [SynGEC: Syntax-Enhanced Grammatical Error Correction with a Tailored GEC-Oriented Parser](https://mlsanity.com/paper/2210.12484/) (2022)
- [On the application of Large Language Models for language teaching and assessment technology](https://mlsanity.com/paper/2307.08393/) (2023)
- [GrammarGPT: Exploring Open-Source LLMs for Native Chinese Grammatical Error Correction with Supervised Fine-Tuning](https://mlsanity.com/paper/2307.13923/) (2023)
- [Advancements in Arabic Grammatical Error Detection and Correction: An Empirical Investigation](https://mlsanity.com/paper/2305.14734/) (2023)
- [UA-GEC: Grammatical Error Correction and Fluency Corpus for the Ukrainian Language](https://mlsanity.com/paper/2103.16997/) (2021)
- [Revisiting Grammatical Error Correction Evaluation and Beyond](https://mlsanity.com/paper/2211.01635/) (2022)
- [A Unified Strategy for Multilingual Grammatical Error Correction with Pre-trained Cross-Lingual Language Model](https://mlsanity.com/paper/2201.10707/) (2022)
- [Towards standardizing Korean Grammatical Error Correction: Datasets and Annotation](https://mlsanity.com/paper/2210.14389/) (2022)
- [Rethinking the Roles of Large Language Models in Chinese Grammatical Error Correction](https://mlsanity.com/paper/2402.11420/) (2024)
- [Enhancing Text Editing for Grammatical Error Correction: Arabic as a Case Study](https://mlsanity.com/paper/2503.00985/) (2025)
- [gec-metrics: A Unified Library for Grammatical Error Correction Evaluation](https://mlsanity.com/paper/2505.19388/) (2025)
- [Non-autoregressive Text Editing with Copy-aware Latent Alignments](https://mlsanity.com/paper/2310.07821/) (2023)
- [Improving Seq2Seq Grammatical Error Correction via Decoding Interventions](https://mlsanity.com/paper/2310.14534/) (2023)
- [From Spelling to Grammar: A New Framework for Chinese Grammatical Error Correction](https://mlsanity.com/paper/2211.01625/) (2022)
- [Rethinking Evaluation Metrics for Grammatical Error Correction: Why Use a Different Evaluation Process than Human?](https://mlsanity.com/paper/2502.09416/) (2025)
- [RobustGEC: Robust Grammatical Error Correction Against Subtle Context Perturbation](https://mlsanity.com/paper/2310.07299/) (2023)
- [Opportunities and Challenges of LLMs in Education: An NLP Perspective](https://mlsanity.com/paper/2507.22753/) (2025)
- [Efficient Grammatical Error Correction Via Multi-Task Training and Optimized Training Schedule](https://mlsanity.com/paper/2311.11813/) (2023)
- [GEC-DePenD: Non-Autoregressive Grammatical Error Correction with Decoupled Permutation and Decoding](https://mlsanity.com/paper/2311.08191/) (2023)
- [MixEdit: Revisiting Data Augmentation and Beyond for Grammatical Error Correction](https://mlsanity.com/paper/2310.11671/) (2023)
- [Beyond Hard Samples: Robust and Effective Grammatical Error Correction with Cycle Self-Augmenting](https://mlsanity.com/paper/2310.13321/) (2023)
- [System Combination via Quality Estimation for Grammatical Error Correction](https://mlsanity.com/paper/2310.14947/) (2023)
- [Leveraging What's Overfixed: Post-Correction via LLM Grammatical Error Overcorrection](https://mlsanity.com/paper/2509.20811/) (2025)
- [Detection-Correction Structure via General Language Model for Grammatical Error Correction](https://mlsanity.com/paper/2405.17804/) (2024)
- [Explanation based In-Context Demonstrations Retrieval for Multilingual Grammatical Error Correction](https://mlsanity.com/paper/2502.08507/) (2025)
- [Leveraging Denoised Abstract Meaning Representation for Grammatical Error Correction](https://mlsanity.com/paper/2307.02127/) (2023)

## Abstract (arXiv)

Studies on grammatical error correction (GEC) have reported the effectiveness of pretraining a Seq2Seq model with a large amount of pseudodata. However, this approach requires time-consuming pretraining for GEC because of the size of the pseudodata. In this study, we explore the utility of bidirectional and auto-regressive transformers (BART) as a generic pretrained encoder-decoder model for GEC. With the use of this generic pretrained model for GEC, the time-consuming pretraining can be eliminated. We find that monolingual and multilingual BART models achieve high performance in GEC, with one of the results being comparable to the current strong results in English GEC. Our implementations are publicly available at GitHub (https://github.com/Katsumata420/generic-pretrained-GEC).