Larger Context Window, Fewer Overcorrections: Optimizing Prompts and Batching for Minimal-Edit Grammatical Error Correction
Not on the map. Why?
Abstract (arXiv)
Minimal-edit Grammatical Error Correction (GEC) is a challenging task for zero- and few-shot prompted Large Language Models (LLMs), which systematically overcorrect and degrade F0.5 by rewriting well-formed spans. While fine-tuning provides an effective solution, it imposes substantial infrastructure demands. We introduce a prompt-based approach that closes the gap to fine-tuned models through three advances in GEC prompting methodology. First, we introduce taxonomy-based instructions to enforce minimal-edit constraints with a comprehensive list of grammatical error rules, equipping the LLM with a bounded, metric-aligned scope of correctable edits, which benefits the strongest models while remaining model-dependent overall. Second, we show that batching multiple uncorrected sentences into a single input context acts as a targeted regularizer against overcorrection, systematically reducing the edit rate across diverse LLM families; we hypothesize this arises from attention dilution effect induced by the bounded capacity of self-attention scores. Finally, LLM-assisted Prompt Optimization refines these instructions. Powered by Gemini 3.1-Pro, our prompt achieves F0.5=78.32 on the BEA-2019 test set - establishing a new prompt-based SOTA while shrinking the gap to the fine-tuned single-model SOTA (Staruch et al., 2025) to a mere 0.38 points. Code, prompts, and outputs are publicly available.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 77 references that are in the map. First posted on arXiv 9 Sep 2026.
Builds on
28 of 77 references.
References · 28
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
- 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
- 2023PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimizationcited by 233
- 2020GECToR -- Grammatical Error Correction: Tag, Not Rewritecited by 177
- 2021A Simple Recipe for Multilingual Grammatical Error Correctioncited by 108
- 2022The Devil in Linear Transformercited by 98
- 2017JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correctioncited by 97
- 2023Is ChatGPT a Highly Fluent Grammatical Error Correction System? A Comprehensive Evaluationcited by 97
- 2023Batch Prompting: Efficient Inference with Large Language Model APIscited by 85
- 2019Parallel Iterative Edit Models for Local Sequence Transductioncited by 81
- 2023Exposing Attention Glitches with Flip-Flop Language Modelingcited by 75
16 more
- 2025A Systematic Survey of Automatic Prompt Optimization Techniquescited by 63
- 2018Approaching Neural Grammatical Error Correction as a Low-Resource Machine Translation Taskcited by 56
- 2023Exploring Effectiveness of GPT-3 in Grammatical Error Correction: A Study on Performance and Controllability in Prompt-Based Methodscited by 37
- 2024Prompting open-source and commercial language models for grammatical error correction of English learner textcited by 24
- 2021UA-GEC: Grammatical Error Correction and Fluency Corpus for the Ukrainian Languagecited by 19
- 2024GPT-3.5 for Grammatical Error Correctioncited by 18
- 2024CriSPO: Multi-Aspect Critique-Suggestion-guided Automatic Prompt Optimization for Text Generationcited by 18
- 2020Personalizing Grammatical Error Correction: Adaptation to Proficiency Level and L1cited by 10
- 2024Evaluating Prompting Strategies for Grammatical Error Correction Based on Language Proficiencycited by 9
- 2025gec-metrics: A Unified Library for Grammatical Error Correction Evaluationcited by 9
- 2025Opportunities and Challenges of LLMs in Education: An NLP Perspectivecited by 6
- 2025Adapting LLMs for Minimal-edit Grammatical Error Correctioncited by 5
- 2023Metric-Based In-context Learning: A Case Study in Text Simplificationcited by 3
- 2025APIO: Automatic Prompt Induction and Optimization for Grammatical Error Correction and Text Simplificationcited by 2
- 2026Edit-level Majority Voting Mitigates Over-Correction in LLM-based Grammatical Error Correctioncited by 1
- 2026How Far Can Prompting Go for Minimal-Edit Ukrainian Grammatical Error Correction?cited by 1
Built on by
Not cited yet.