# Data Weighted Training Strategies for Grammatical Error Correction

arXiv: https://arxiv.org/abs/2008.02976 · HTML: https://mlsanity.com/paper/2008.02976/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Aug 2020.

## Built on by (most cited first, 26 of 26)

- [A Simple Recipe for Multilingual Grammatical Error Correction](https://mlsanity.com/paper/2106.03830/) (2021)
- [Synthetic Data Generation for Grammatical Error Correction with Tagged Corruption Models](https://mlsanity.com/paper/2105.13318/) (2021)
- [Grammatical Error Correction: A Survey of the State of the Art](https://mlsanity.com/paper/2211.05166/) (2022)
- [SynGEC: Syntax-Enhanced Grammatical Error Correction with a Tailored GEC-Oriented Parser](https://mlsanity.com/paper/2210.12484/) (2022)
- [GPT-3.5 for Grammatical Error Correction](https://mlsanity.com/paper/2405.08469/) (2024)
- [Czech Grammar Error Correction with a Large and Diverse Corpus](https://mlsanity.com/paper/2201.05590/) (2022)
- [Modeling Annotator Disagreement with Demographic-Aware Experts and Synthetic Perspectives](https://mlsanity.com/paper/2508.02853/) (2025)
- [Uncertainty Determines the Adequacy of the Mode and the Tractability of Decoding in Sequence-to-Sequence Models](https://mlsanity.com/paper/2204.00471/) (2022)
- [Efficient Grammatical Error Correction Via Multi-Task Training and Optimized Training Schedule](https://mlsanity.com/paper/2311.11813/) (2023)
- [CSynGEC: Incorporating Constituent-based Syntax for Grammatical Error Correction with a Tailored GEC-Oriented Parser](https://mlsanity.com/paper/2211.08158/) (2022)
- [Minimum Bayes' Risk Decoding for System Combination of Grammatical Error Correction Systems](https://mlsanity.com/paper/2309.06520/) (2023)
- [GEC-DePenD: Non-Autoregressive Grammatical Error Correction with Decoupled Permutation and Decoding](https://mlsanity.com/paper/2311.08191/) (2023)
- [Loss-Aware Curriculum Learning for Chinese Grammatical Error Correction](https://mlsanity.com/paper/2501.00334/) (2024)
- [MixEdit: Revisiting Data Augmentation and Beyond for Grammatical Error Correction](https://mlsanity.com/paper/2310.11671/) (2023)
- [ChatLang-8: An LLM-Based Synthetic Data Generation Framework for Grammatical Error Correction](https://mlsanity.com/paper/2406.03202/) (2024)
- [Synthesizing and Adapting Error Correction Data for Mobile Large Language Model Applications](https://mlsanity.com/paper/2505.18488/) (2025)
- [Detecting over/under-translation errors for determining adequacy in human translations](https://mlsanity.com/paper/2104.00267/) (2021)
- [Abstraction not Memory: BERT and the English Article System](https://mlsanity.com/paper/2206.04184/) (2022)
- [A BERT-based Unsupervised Grammatical Error Correction Framework](https://mlsanity.com/paper/2303.17367/) (2023)
- [RedPenNet for Grammatical Error Correction: Outputs to Tokens, Attentions to Spans](https://mlsanity.com/paper/2309.10898/) (2023)
- [Grammatical Error Correction via Mixed-Grained Weighted Training](https://mlsanity.com/paper/2311.13848/) (2023)
- [Detection-Correction Structure via General Language Model for Grammatical Error Correction](https://mlsanity.com/paper/2405.17804/) (2024)
- [Assessing the Efficacy of Grammar Error Correction: A Human Evaluation Approach in the Japanese Context](https://mlsanity.com/paper/2402.18101/) (2024)
- [Multi-head Sequence Tagging Model for Grammatical Error Correction](https://mlsanity.com/paper/2410.16473/) (2024)
- [Harnessing Rule-Based Reinforcement Learning for Enhanced Grammatical Error Correction](https://mlsanity.com/paper/2508.18780/) (2025)
- [RILEC: Detection and Generation of L1 Russian Interference Errors in English Learner Texts](https://mlsanity.com/paper/2603.07366/) (2026)

## Abstract (arXiv)

Recent progress in the task of Grammatical Error Correction (GEC) has been driven by addressing data sparsity, both through new methods for generating large and noisy pretraining data and through the publication of small and higher-quality finetuning data in the BEA-2019 shared task. Building upon recent work in Neural Machine Translation (NMT), we make use of both kinds of data by deriving example-level scores on our large pretraining data based on a smaller, higher-quality dataset. In this work, we perform an empirical study to discover how to best incorporate delta-log-perplexity, a type of example scoring, into a training schedule for GEC. In doing so, we perform experiments that shed light on the function and applicability of delta-log-perplexity. Models trained on scored data achieve state-of-the-art results on common GEC test sets.