# Improving Seq2Seq Grammatical Error Correction via Decoding Interventions

arXiv: https://arxiv.org/abs/2310.14534 · HTML: https://mlsanity.com/paper/2310.14534/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 38 references that are in the map.
- First posted on arXiv 23 Oct 2023.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts](https://mlsanity.com/paper/2105.03023/) (2021)
- [GeDi: Generative Discriminator Guided Sequence Generation](https://mlsanity.com/paper/2009.06367/) (2020)
- [Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration](https://mlsanity.com/paper/1910.12656/) (2019)
- [GECToR -- Grammatical Error Correction: Tag, Not Rewrite](https://mlsanity.com/paper/2005.12592/) (2020)
- [On Using Monolingual Corpora in Neural Machine Translation](https://mlsanity.com/paper/1503.03535/) (2015)
- [An analysis of incorporating an external language model into a sequence-to-sequence model](https://mlsanity.com/paper/1712.01996/) (2017)
- [A Simple Recipe for Multilingual Grammatical Error Correction](https://mlsanity.com/paper/2106.03830/) (2021)
- [Parallel Iterative Edit Models for Local Sequence Transduction](https://mlsanity.com/paper/1910.02893/) (2019)
- [Synthetic Data Generation for Grammatical Error Correction with Tagged Corruption Models](https://mlsanity.com/paper/2105.13318/) (2021)
- [Instantaneous Grammatical Error Correction with Shallow Aggressive Decoding](https://mlsanity.com/paper/2106.04970/) (2021)
- [FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognition](https://mlsanity.com/paper/2105.03842/) (2021)
- [Stronger Baselines for Grammatical Error Correction Using Pretrained Encoder-Decoder Model](https://mlsanity.com/paper/2005.11849/) (2020)
- [LM-Critic: Language Models for Unsupervised Grammatical Error Correction](https://mlsanity.com/paper/2109.06822/) (2021)
- [SynGEC: Syntax-Enhanced Grammatical Error Correction with a Tailored GEC-Oriented Parser](https://mlsanity.com/paper/2210.12484/) (2022)
- [Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction](https://mlsanity.com/paper/2010.03260/) (2020)
- [Controllable Text Generation with Language Constraints](https://mlsanity.com/paper/2212.10466/) (2022)
- [Non-autoregressive Text Editing with Copy-aware Latent Alignments](https://mlsanity.com/paper/2310.07821/) (2023)
- [Do Grammatical Error Correction Models Realize Grammatical Generalization?](https://mlsanity.com/paper/2106.03031/) (2021)

## Built on by (most cited first, 8 of 8)

- [Prompting open-source and commercial language models for grammatical error correction of English learner text](https://mlsanity.com/paper/2401.07702/) (2024)
- [Enhancing Text Editing for Grammatical Error Correction: Arabic as a Case Study](https://mlsanity.com/paper/2503.00985/) (2025)
- [Opportunities and Challenges of LLMs in Education: An NLP Perspective](https://mlsanity.com/paper/2507.22753/) (2025)
- [Mixture of Small and Large Models for Chinese Spelling Check](https://mlsanity.com/paper/2506.06887/) (2025)
- [Detection-Correction Structure via General Language Model for Grammatical Error Correction](https://mlsanity.com/paper/2405.17804/) (2024)
- [Zero-shot Cross-Lingual Transfer for Synthetic Data Generation in Grammatical Error Detection](https://mlsanity.com/paper/2407.11854/) (2024)
- [JELV: A Judge of Edit-Level Validity for Evaluation and Automated Reference Expansion in Grammatical Error Correction](https://mlsanity.com/paper/2511.21700/) (2025)
- [Encode Errors: Representational Retrieval of In-Context Demonstrations for Multilingual Grammatical Error Correction](https://mlsanity.com/paper/2606.15416/) (2026)

## Abstract (arXiv)

The sequence-to-sequence (Seq2Seq) approach has recently been widely used in grammatical error correction (GEC) and shows promising performance. However, the Seq2Seq GEC approach still suffers from two issues. First, a Seq2Seq GEC model can only be trained on parallel data, which, in GEC task, is often noisy and limited in quantity. Second, the decoder of a Seq2Seq GEC model lacks an explicit awareness of the correctness of the token being generated. In this paper, we propose a unified decoding intervention framework that employs an external critic to assess the appropriateness of the token to be generated incrementally, and then dynamically influence the choice of the next token. We discover and investigate two types of critics: a pre-trained left-to-right language model critic and an incremental target-side grammatical error detector critic. Through extensive experiments on English and Chinese datasets, our framework consistently outperforms strong baselines and achieves results competitive with state-of-the-art methods.