# Prompting open-source and commercial language models for grammatical error correction of English learner text

arXiv: https://arxiv.org/abs/2401.07702 · HTML: https://mlsanity.com/paper/2401.07702/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 56 references that are in the map.
- First posted on arXiv 15 Jan 2024.

## Builds on (most cited first, 24 of 24)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [How Can We Know What Language Models Know?](https://mlsanity.com/paper/1911.12543/) (2019)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [The Internal State of an LLM Knows When It's Lying](https://mlsanity.com/paper/2304.13734/) (2023)
- [OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization](https://mlsanity.com/paper/2212.12017/) (2022)
- ["Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters](https://mlsanity.com/paper/2310.09219/) (2023)
- [GECToR -- Grammatical Error Correction: Tag, Not Rewrite](https://mlsanity.com/paper/2005.12592/) (2020)
- [The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediations](https://mlsanity.com/paper/2310.04988/) (2023)
- [A Simple Recipe for Multilingual Grammatical Error Correction](https://mlsanity.com/paper/2106.03830/) (2021)
- [JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correction](https://mlsanity.com/paper/1702.04066/) (2017)
- [Is ChatGPT a Highly Fluent Grammatical Error Correction System? A Comprehensive Evaluation](https://mlsanity.com/paper/2304.01746/) (2023)
- [ChatGPT or Grammarly? Evaluating ChatGPT on Grammatical Error Correction Benchmark](https://mlsanity.com/paper/2303.13648/) (2023)
- [WikiChat: Stopping the Hallucination of Large Language Model Chatbots by Few-Shot Grounding on Wikipedia](https://mlsanity.com/paper/2305.14292/) (2023)
- [Seq2Edits: Sequence Transduction Using Span-level Edit Operations](https://mlsanity.com/paper/2009.11136/) (2020)
- [Approaching Neural Grammatical Error Correction as a Low-Resource Machine Translation Task](https://mlsanity.com/paper/1804.05940/) (2018)
- [Synthetic Data Generation for Grammatical Error Correction with Tagged Corruption Models](https://mlsanity.com/paper/2105.13318/) (2021)
- [Improving Diversity of Demographic Representation in Large Language Models via Collective-Critiques and Self-Voting](https://mlsanity.com/paper/2310.16523/) (2023)
- [Exploring Effectiveness of GPT-3 in Grammatical Error Correction: A Study on Performance and Controllability in Prompt-Based Methods](https://mlsanity.com/paper/2305.18156/) (2023)
- [LM-Critic: Language Models for Unsupervised Grammatical Error Correction](https://mlsanity.com/paper/2109.06822/) (2021)
- [Language Models Get a Gender Makeover: Mitigating Gender Bias with Few-Shot Data Interventions](https://mlsanity.com/paper/2306.04597/) (2023)
- [On the application of Large Language Models for language teaching and assessment technology](https://mlsanity.com/paper/2307.08393/) (2023)
- [Why LLMs Hallucinate, and How to Get (Evidential) Closure: Perceptual, Intensional, and Extensional Learning for Faithful Natural Language Generation](https://mlsanity.com/paper/2310.15355/) (2023)
- [Neural Quality Estimation with Multiple Hypotheses for Grammatical Error Correction](https://mlsanity.com/paper/2105.04443/) (2021)
- [Improving Seq2Seq Grammatical Error Correction via Decoding Interventions](https://mlsanity.com/paper/2310.14534/) (2023)

## Built on by (most cited first, 24 of 24)

- [Rethinking the Roles of Large Language Models in Chinese Grammatical Error Correction](https://mlsanity.com/paper/2402.11420/) (2024)
- [Enhancing Text Editing for Grammatical Error Correction: Arabic as a Case Study](https://mlsanity.com/paper/2503.00985/) (2025)
- [Metaheuristics and Large Language Models Join Forces: Toward an Integrated Optimization Approach](https://mlsanity.com/paper/2405.18272/) (2024)
- [Opportunities and Challenges of LLMs in Education: An NLP Perspective](https://mlsanity.com/paper/2507.22753/) (2025)
- [Adapting LLMs for Minimal-edit Grammatical Error Correction](https://mlsanity.com/paper/2506.13148/) (2025)
- [Proofread: Fixes All Errors with One Tap](https://mlsanity.com/paper/2406.04523/) (2024)
- [Leveraging LLM For Synchronizing Information Across Multilingual Tables](https://mlsanity.com/paper/2504.02559/) (2025)
- [Exploring the Feasibility of Multilingual Grammatical Error Correction with a Single LLM up to 9B parameters: A Comparative Study of 17 Models](https://mlsanity.com/paper/2505.06004/) (2025)
- [APIO: Automatic Prompt Induction and Optimization for Grammatical Error Correction and Text Simplification](https://mlsanity.com/paper/2508.09378/) (2025)
- [Trigger3: Refining Query Correction via Adaptive Model Selector](https://mlsanity.com/paper/2412.12701/) (2024)
- [Explanation based In-Context Demonstrations Retrieval for Multilingual Grammatical Error Correction](https://mlsanity.com/paper/2502.08507/) (2025)
- [Corrections Meet Explanations: A Unified Framework for Explainable Grammatical Error Correction](https://mlsanity.com/paper/2502.15261/) (2025)
- [Edit-level Majority Voting Mitigates Over-Correction in LLM-based Grammatical Error Correction](https://mlsanity.com/paper/2605.13624/) (2026)
- [Zero-shot Cross-Lingual Transfer for Synthetic Data Generation in Grammatical Error Detection](https://mlsanity.com/paper/2407.11854/) (2024)
- [KSOD: Knowledge Supplement for LLMs On Demand](https://mlsanity.com/paper/2503.07550/) (2025)
- [Auto Review: Second Stage Error Detection for Highly Accurate Information Extraction from Phone Conversations](https://mlsanity.com/paper/2506.05400/) (2025)
- [Time Is Effort: Estimating Human Post-Editing Time for Grammar Error Correction Tool Evaluation](https://mlsanity.com/paper/2510.04394/) (2025)
- [Sandwich Reasoning: An Answer-Reasoning-Answer Approach for Low-Latency Query Correction](https://mlsanity.com/paper/2601.03672/) (2026)
- [Incentives Of EdTech: A Systematic Review Of EduNLP Research](https://mlsanity.com/paper/2606.13691/) (2026)
- [Encode Errors: Representational Retrieval of In-Context Demonstrations for Multilingual Grammatical Error Correction](https://mlsanity.com/paper/2606.15416/) (2026)
- [WrAFT: a Modularized Automated Writing Evaluation System for Argumentative Essays](https://mlsanity.com/paper/2607.14524/) (2026)
- [STAGEET: Stage-wise Typed Edit Tagging for Grammatical Error Correction with Arabic as a Case Study](https://mlsanity.com/paper/2608.28614/) (2026)
- [How Do Language Models Represent and Use Phonological Information for Allomorph Selection?](https://mlsanity.com/paper/2609.04708/) (2026)
- [Larger Context Window, Fewer Overcorrections: Optimizing Prompts and Batching for Minimal-Edit Grammatical Error Correction](https://mlsanity.com/paper/2609.10810/) (2026)

## Abstract (arXiv)

Thanks to recent advances in generative AI, we are able to prompt large language models (LLMs) to produce texts which are fluent and grammatical. In addition, it has been shown that we can elicit attempts at grammatical error correction (GEC) from LLMs when prompted with ungrammatical input sentences. We evaluate how well LLMs can perform at GEC by measuring their performance on established benchmark datasets. We go beyond previous studies, which only examined GPT* models on a selection of English GEC datasets, by evaluating seven open-source and three commercial LLMs on four established GEC benchmarks. We investigate model performance and report results against individual error types. Our results indicate that LLMs do not always outperform supervised English GEC models except in specific contexts -- namely commercial LLMs on benchmarks annotated with fluency corrections as opposed to minimal edits. We find that several open-source models outperform commercial ones on minimal edit benchmarks, and that in some settings zero-shot prompting is just as competitive as few-shot prompting.