mlsanityData to 14 Oct 2026

Synthetic Data Generation for Grammatical Error Correction with Tagged Corruption Models

arXiv 2105.13318 ↗First posted 27 May 2021cs.CLcited by 56

Not on the map. Why?

Abstract (arXiv)

Synthetic data generation is widely known to boost the accuracy of neural grammatical error correction (GEC) systems, but existing methods often lack diversity or are too simplistic to generate the broad range of grammatical errors made by human writers. In this work, we use error type tags from automatic annotation tools such as ERRANT to guide synthetic data generation. We compare several models that can produce an ungrammatical sentence given a clean sentence and an error type tag. We use these models to build a new, large synthetic pre-training data set with error tag frequency distributions matching a given development set. Our synthetic data set yields large and consistent gains, improving the state-of-the-art on the BEA-19 and CoNLL-14 test sets. We also show that our approach is particularly effective in adapting a GEC system, trained on mixed native and non-native English, to a native English test set, even surpassing real training data consisting of high-quality sentence pairs.

Key facts. Cited by 56 papers in the map (data through 14 Oct 2026). Builds on 22 of its 48 references that are in the map. First posted on arXiv 27 May 2021.

Builds on

22 of 48 references.

References · 22

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 905
  4. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
  5. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 469
  6. 2020GECToR -- Grammatical Error Correction: Tag, Not Rewritecited by 177
  7. 2017JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correctioncited by 97
  8. 2018Tensor2Tensor for Neural Machine Translationcited by 85
  9. 2019Tagged Back-Translationcited by 77
  10. 2020Encoder-Decoder Models Can Benefit from Pre-trained Masked Language Models in Grammatical Error Correctioncited by 59
  11. 2020Seq2Edits: Sequence Transduction Using Span-level Edit Operationscited by 58
  12. 2018Approaching Neural Grammatical Error Correction as a Low-Resource Machine Translation Taskcited by 56
10 more
  1. 2019An Empirical Study of Incorporating Pseudo Data into Grammatical Error Correctioncited by 55
  2. 2019Corpora Generation for Grammatical Error Correctioncited by 53
  3. 2019Improving Grammatical Error Correction via Pre-Training a Copy-Augmented Architecture with Unlabeled Datacited by 50
  4. 2018A Multilayer Convolutional Encoder-Decoder Neural Network for Grammatical Error Correctioncited by 36
  5. 2019A Neural Grammatical Error Correction System Built On Better Pre-training and Sequential Transfer Learningcited by 26
  6. 2020Data Weighted Training Strategies for Grammatical Error Correctioncited by 26
  7. 2018Reaching Human-level Performance in Automatic Grammatical Error Correction: An Empirical Studycited by 22
  8. 2017Artificial Error Generation with Machine Translation and Syntactic Patternscited by 11
  9. 2018Wronging a Right: Generating Better Errors to Improve Grammatical Error Detectioncited by 11
  10. 2019The CUED's Grammatical Error Correction Systems for BEA-2019cited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 15; 2023: 16; 2024: 13; 2025: 8; 2026: 2.

56 papers.

Sort:

By area:

  1. 2023Is ChatGPT a Highly Fluent Grammatical Error Correction System? A Comprehensive Evaluationcited by 97
  2. 2022Grammatical Error Correction: A Survey of the State of the Artcited by 43
  3. 2022EdiT5: Semi-Autoregressive Text-Editing with T5 Warm-Startcited by 35
  4. 2023Multi-Task Instruction Tuning of LLaMa for Specific Scenarios: A Preliminary Study on Writing Assistancecited by 33
  5. 2022SynGEC: Syntax-Enhanced Grammatical Error Correction with a Tailored GEC-Oriented Parsercited by 32
  6. 2024Prompting open-source and commercial language models for grammatical error correction of English learner textcited by 24
  7. 2022Text Generation with Text-Editing Modelscited by 21
  8. 2025Bridging the Editing Gap in LLMs: FineEdit for Precise and Targeted Text Modificationscited by 21
  9. 2024Resilience of Large Language Models for Noisy Instructionscited by 20
  10. 2023Enhancing Grammatical Error Correction Systems with Explanationscited by 19
  11. 2022A Unified Strategy for Multilingual Grammatical Error Correction with Pre-trained Cross-Lingual Language Modelcited by 16
  12. 2024Reconstruct Your Previous Conversations! Comprehensively Investigating Privacy Leakage Risks in Conversations with GPT Modelscited by 16
38 more
  1. 2022Ensembling and Knowledge Distilling of Large Sequence Taggers for Grammatical Error Correctioncited by 12
  2. 2024Robustness of Large Language Models to Perturbations in Textcited by 11
  3. 2022Uncertainty Determines the Adequacy of the Mode and the Tractability of Decoding in Sequence-to-Sequence Modelscited by 9
  4. 2025Enhancing Text Editing for Grammatical Error Correction: Arabic as a Case Studycited by 9
  5. 2023Improving Seq2Seq Grammatical Error Correction via Decoding Interventionscited by 8
  6. 2022User-Centric Gender Rewritingcited by 7
  7. 2023Analyzing the Performance of GPT-3.5 and GPT-4 in Grammatical Error Correctioncited by 7
  8. 2023Byte-Level Grammatical Error Correction Using Synthetic and Curated Corporacited by 7
  9. 2023RobustGEC: Robust Grammatical Error Correction Against Subtle Context Perturbationcited by 6
  10. 2024Improving Grammatical Error Correction via Contextual Data Augmentationcited by 6
  11. 2023Does Collaborative Human-LM Dialogue Generation Help Information Extraction from Human Dialogues?cited by 5
  12. 2023Efficient Grammatical Error Correction Via Multi-Task Training and Optimized Training Schedulecited by 5
  13. 2025Adapting LLMs for Minimal-edit Grammatical Error Correctioncited by 5
  14. 2022CSynGEC: Incorporating Constituent-based Syntax for Grammatical Error Correction with a Tailored GEC-Oriented Parsercited by 4
  15. 2024LLM-based Code-Switched Text Generation for Grammatical Error Correctioncited by 4
  16. 2021Diversity-Driven Combination for Grammatical Error Correctioncited by 3
  17. 2022BSpell: A CNN-Blended BERT Based Bangla Spell Checkercited by 3
  18. 2023GEC-DePenD: Non-Autoregressive Grammatical Error Correction with Decoupled Permutation and Decodingcited by 3
  19. 2024To Err Is Human, but Llamas Can Learn It Toocited by 3
  20. 2021ErAConD : Error Annotated Conversational Dialog Dataset for Grammatical Error Correctioncited by 2
  21. 2022Knowledge-Grounded Conversational Data Augmentation with Generative Conversational Networkscited by 2
  22. 2023MixEdit: Revisiting Data Augmentation and Beyond for Grammatical Error Correctioncited by 2
  23. 2023System Combination via Quality Estimation for Grammatical Error Correctioncited by 2
  24. 2024Grammatical Error Correction for Code-Switched Sentences by Learners of Englishcited by 2
  25. 2024ChatLang-8: An LLM-Based Synthetic Data Generation Framework for Grammatical Error Correctioncited by 2
  26. 2025Synthesizing and Adapting Error Correction Data for Mobile Large Language Model Applicationscited by 2
  27. 2025Leveraging What's Overfixed: Post-Correction via LLM Grammatical Error Overcorrectioncited by 2
  28. 2022Judge a Sentence by Its Content to Generate Grammatical Errorscited by 1
  29. 2023Bidirectional Transformer Reranker for Grammatical Error Correctioncited by 1
  30. 2023RedPenNet for Grammatical Error Correction: Outputs to Tokens, Attentions to Spanscited by 1
  31. 2023Grammatical Error Correction via Mixed-Grained Weighted Trainingcited by 1
  32. 2024Detection-Correction Structure via General Language Model for Grammatical Error Correctioncited by 1
  33. 2025Cyborg Data: Merging Human with AI Generated Training Datacited by 1
  34. 2022Proficiency Matters Quality Estimation in Grammatical Error Correctioncited by 0
  35. 2022Evaluation of large-scale synthetic data for Grammar Error Correctioncited by 0
  36. 2022Assistive Completion of Agrammatic Aphasic Sentences: A Transfer Learning Approach using Neurolinguistics-based Synthetic Datasetcited by 0
  37. 2023Wiki-En-ASR-Adapt: Large-scale synthetic dataset for English ASR Customizationcited by 0
  38. 2024Neural Automated Writing Evaluation with Corrective Feedbackcited by 0

50 most cited of 56 All 56 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Unveiling the Flaws: Exploring Imperfections in Synthetic Data and Mitigation Strategies for Large Language Models
  2. 2024Efficient and Interpretable Grammatical Error Correction with Mixture of Experts
  3. 2025Synthetic Data Generation Using Large Language Models: Advances in Text and Code
  4. 2024Evaluating Language Models as Synthetic Data Generators
  5. 2024An Empirical Study of Validating Synthetic Data for Formula Generation
  6. 2024Leveraging LLMs for Bangla Grammar Error Correction:Error Categorization, Synthetic Data, and Model Evaluation
  7. 2026How Can We Synthesize High-Quality Pretraining Data? A Systematic Study of Prompt Design, Generator Model, and Source Data
  8. 2023Generating Faithful Synthetic Data with Large Language Models: A Case Study in Computational Social Science