# Larger Context Window, Fewer Overcorrections: Optimizing Prompts and Batching for Minimal-Edit Grammatical Error Correction

arXiv: https://arxiv.org/abs/2609.10810 · HTML: https://mlsanity.com/paper/2609.10810/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 77 references that are in the map.
- First posted on arXiv 9 Sep 2026.

## Builds on (most cited first, 28 of 28)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Automatic Prompt Optimization with "Gradient Descent" and Beam Search](https://mlsanity.com/paper/2305.03495/) (2023)
- [PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization](https://mlsanity.com/paper/2310.16427/) (2023)
- [GECToR -- Grammatical Error Correction: Tag, Not Rewrite](https://mlsanity.com/paper/2005.12592/) (2020)
- [A Simple Recipe for Multilingual Grammatical Error Correction](https://mlsanity.com/paper/2106.03830/) (2021)
- [The Devil in Linear Transformer](https://mlsanity.com/paper/2210.10340/) (2022)
- [JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correction](https://mlsanity.com/paper/1702.04066/) (2017)
- [Is ChatGPT a Highly Fluent Grammatical Error Correction System? A Comprehensive Evaluation](https://mlsanity.com/paper/2304.01746/) (2023)
- [Batch Prompting: Efficient Inference with Large Language Model APIs](https://mlsanity.com/paper/2301.08721/) (2023)
- [Parallel Iterative Edit Models for Local Sequence Transduction](https://mlsanity.com/paper/1910.02893/) (2019)
- [Exposing Attention Glitches with Flip-Flop Language Modeling](https://mlsanity.com/paper/2306.00946/) (2023)
- [A Systematic Survey of Automatic Prompt Optimization Techniques](https://mlsanity.com/paper/2502.16923/) (2025)
- [Approaching Neural Grammatical Error Correction as a Low-Resource Machine Translation Task](https://mlsanity.com/paper/1804.05940/) (2018)
- [Exploring Effectiveness of GPT-3 in Grammatical Error Correction: A Study on Performance and Controllability in Prompt-Based Methods](https://mlsanity.com/paper/2305.18156/) (2023)
- [Prompting open-source and commercial language models for grammatical error correction of English learner text](https://mlsanity.com/paper/2401.07702/) (2024)
- [UA-GEC: Grammatical Error Correction and Fluency Corpus for the Ukrainian Language](https://mlsanity.com/paper/2103.16997/) (2021)
- [GPT-3.5 for Grammatical Error Correction](https://mlsanity.com/paper/2405.08469/) (2024)
- [CriSPO: Multi-Aspect Critique-Suggestion-guided Automatic Prompt Optimization for Text Generation](https://mlsanity.com/paper/2410.02748/) (2024)
- [Personalizing Grammatical Error Correction: Adaptation to Proficiency Level and L1](https://mlsanity.com/paper/2006.02964/) (2020)
- [Evaluating Prompting Strategies for Grammatical Error Correction Based on Language Proficiency](https://mlsanity.com/paper/2402.15930/) (2024)
- [gec-metrics: A Unified Library for Grammatical Error Correction Evaluation](https://mlsanity.com/paper/2505.19388/) (2025)
- [Opportunities and Challenges of LLMs in Education: An NLP Perspective](https://mlsanity.com/paper/2507.22753/) (2025)
- [Adapting LLMs for Minimal-edit Grammatical Error Correction](https://mlsanity.com/paper/2506.13148/) (2025)
- [Metric-Based In-context Learning: A Case Study in Text Simplification](https://mlsanity.com/paper/2307.14632/) (2023)
- [APIO: Automatic Prompt Induction and Optimization for Grammatical Error Correction and Text Simplification](https://mlsanity.com/paper/2508.09378/) (2025)
- [Edit-level Majority Voting Mitigates Over-Correction in LLM-based Grammatical Error Correction](https://mlsanity.com/paper/2605.13624/) (2026)
- [How Far Can Prompting Go for Minimal-Edit Ukrainian Grammatical Error Correction?](https://mlsanity.com/paper/2606.09334/) (2026)

## Abstract (arXiv)

Minimal-edit Grammatical Error Correction (GEC) is a challenging task for zero- and few-shot prompted Large Language Models (LLMs), which systematically overcorrect and degrade F0.5 by rewriting well-formed spans. While fine-tuning provides an effective solution, it imposes substantial infrastructure demands. We introduce a prompt-based approach that closes the gap to fine-tuned models through three advances in GEC prompting methodology. First, we introduce taxonomy-based instructions to enforce minimal-edit constraints with a comprehensive list of grammatical error rules, equipping the LLM with a bounded, metric-aligned scope of correctable edits, which benefits the strongest models while remaining model-dependent overall. Second, we show that batching multiple uncorrected sentences into a single input context acts as a targeted regularizer against overcorrection, systematically reducing the edit rate across diverse LLM families; we hypothesize this arises from attention dilution effect induced by the bounded capacity of self-attention scores. Finally, LLM-assisted Prompt Optimization refines these instructions. Powered by Gemini 3.1-Pro, our prompt achieves F0.5=78.32 on the BEA-2019 test set - establishing a new prompt-based SOTA while shrinking the gap to the fine-tuned single-model SOTA (Staruch et al., 2025) to a mere 0.38 points. Code, prompts, and outputs are publicly available.