# Do Grammatical Error Correction Models Realize Grammatical Generalization?

arXiv: https://arxiv.org/abs/2106.03031 · HTML: https://mlsanity.com/paper/2106.03031/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 24 references that are in the map.
- First posted on arXiv 6 Jun 2021.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies](https://mlsanity.com/paper/1611.01368/) (2016)
- [Targeted Syntactic Evaluation of Language Models](https://mlsanity.com/paper/1808.09031/) (2018)
- [Colorless green recurrent networks dream hierarchically](https://mlsanity.com/paper/1803.11138/) (2018)
- [GECToR -- Grammatical Error Correction: Tag, Not Rewrite](https://mlsanity.com/paper/2005.12592/) (2020)
- [Encoder-Decoder Models Can Benefit from Pre-trained Masked Language Models in Grammatical Error Correction](https://mlsanity.com/paper/2005.00987/) (2020)
- [Approaching Neural Grammatical Error Correction as a Low-Resource Machine Translation Task](https://mlsanity.com/paper/1804.05940/) (2018)
- [An Empirical Study of Incorporating Pseudo Data into Grammatical Error Correction](https://mlsanity.com/paper/1909.00502/) (2019)
- [Improving Grammatical Error Correction via Pre-Training a Copy-Augmented Architecture with Unlabeled Data](https://mlsanity.com/paper/1903.00138/) (2019)
- [Do Neural Models Learn Systematicity of Monotonicity Inference in Natural Language?](https://mlsanity.com/paper/2004.14839/) (2020)
- [Do RNNs learn human-like abstract word order preferences?](https://mlsanity.com/paper/1811.01866/) (2018)
- [A Self-Refinement Strategy for Noise Reduction in Grammatical Error Correction](https://mlsanity.com/paper/2010.03155/) (2020)
- [Near Human-Level Performance in Grammatical Error Correction with Hybrid Machine Translation](https://mlsanity.com/paper/1804.05945/) (2018)
- [Cross-Corpora Evaluation and Analysis of Grammatical Error Correction Models --- Is Single-Corpus Evaluation Enough?](https://mlsanity.com/paper/1904.02927/) (2019)

## Built on by (most cited first, 6 of 6)

- [Grammatical Error Correction: A Survey of the State of the Art](https://mlsanity.com/paper/2211.05166/) (2022)
- [Improving Seq2Seq Grammatical Error Correction via Decoding Interventions](https://mlsanity.com/paper/2310.14534/) (2023)
- [FlaCGEC: A Chinese Grammatical Error Correction Dataset with Fine-grained Linguistic Annotation](https://mlsanity.com/paper/2311.04906/) (2023)
- [A Comprehensive Evaluation and Analysis Study for Chinese Spelling Check](https://mlsanity.com/paper/2307.13655/) (2023)
- [Exploring the Capacity of a Large-scale Masked Language Model to Recognize Grammatical Errors](https://mlsanity.com/paper/2108.12216/) (2021)
- [Beyond Hard Samples: Robust and Effective Grammatical Error Correction with Cycle Self-Augmenting](https://mlsanity.com/paper/2310.13321/) (2023)

## Abstract (arXiv)

There has been an increased interest in data generation approaches to grammatical error correction (GEC) using pseudo data. However, these approaches suffer from several issues that make them inconvenient for real-world deployment including a demand for large amounts of training data. On the other hand, some errors based on grammatical rules may not necessarily require a large amount of data if GEC models can realize grammatical generalization. This study explores to what extent GEC models generalize grammatical knowledge required for correcting errors. We introduce an analysis method using synthetic and real GEC datasets with controlled vocabularies to evaluate whether models can generalize to unseen errors. We found that a current standard Transformer-based GEC model fails to realize grammatical generalization even in simple settings with limited vocabulary and syntax, suggesting that it lacks the generalization ability required to correct errors from provided training examples.