# Controlled Generation with Prompt Insertion for Natural Language Explanations in Grammatical Error Correction

arXiv: https://arxiv.org/abs/2309.11439 · HTML: https://mlsanity.com/paper/2309.11439/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 42 references that are in the map.
- First posted on arXiv 20 Sep 2023.

## Builds on (most cited first, 20 of 20)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [GECToR -- Grammatical Error Correction: Tag, Not Rewrite](https://mlsanity.com/paper/2005.12592/) (2020)
- [Reframing Human-AI Collaboration for Generating Free-Text Explanations](https://mlsanity.com/paper/2112.08674/) (2021)
- [Explanations from Large Language Models Make Small Reasoners Better](https://mlsanity.com/paper/2210.06726/) (2022)
- [Encode, Tag, Realize: High-Precision Text Editing](https://mlsanity.com/paper/1909.01187/) (2019)
- [Is ChatGPT a Highly Fluent Grammatical Error Correction System? A Comprehensive Evaluation](https://mlsanity.com/paper/2304.01746/) (2023)
- [Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanations](https://mlsanity.com/paper/2307.08678/) (2023)
- [Gender Bias in Masked Language Models for Multiple Languages](https://mlsanity.com/paper/2205.00551/) (2022)
- [Encoder-Decoder Models Can Benefit from Pre-trained Masked Language Models in Grammatical Error Correction](https://mlsanity.com/paper/2005.00987/) (2020)
- [Exploring Effectiveness of GPT-3 in Grammatical Error Correction: A Study on Performance and Controllability in Prompt-Based Methods](https://mlsanity.com/paper/2305.18156/) (2023)
- [Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction](https://mlsanity.com/paper/2010.03260/) (2020)
- [Enhancing Grammatical Error Correction Systems with Explanations](https://mlsanity.com/paper/2305.15676/) (2023)
- [Evaluating Gender Bias of Pre-trained Language Models in Natural Language Inference by Considering All Labels](https://mlsanity.com/paper/2309.09697/) (2023)
- [Interpretability for Language Learners Using Example-Based Grammatical Error Correction](https://mlsanity.com/paper/2203.07085/) (2022)
- [Controllable Data Synthesis Method for Grammatical Error Correction](https://mlsanity.com/paper/1909.13302/) (2019)
- [Comparing Intrinsic Gender Bias Evaluation Measures without using Human Annotated Examples](https://mlsanity.com/paper/2301.12074/) (2023)
- [Solving NLP Problems through Human-System Collaboration: A Discussion-based Approach](https://mlsanity.com/paper/2305.11789/) (2023)
- [Diversity-Driven Combination for Grammatical Error Correction](https://mlsanity.com/paper/2110.15149/) (2021)

## Built on by (most cited first, 7 of 7)

- [Evaluating Gender Bias in Large Language Models via Chain-of-Thought Prompting](https://mlsanity.com/paper/2401.15585/) (2024)
- [Rethinking the Roles of Large Language Models in Chinese Grammatical Error Correction](https://mlsanity.com/paper/2402.11420/) (2024)
- [Imitating Mistakes in a Learning Companion AI Agent for Online Peer Learning](https://mlsanity.com/paper/2507.12801/) (2025)
- [Scaling and Prompting for Improved End-to-End Spoken Grammatical Error Correction](https://mlsanity.com/paper/2505.21137/) (2025)
- [Annotating Errors in English Learners' Written Language Production: Advancing Automated Written Feedback Systems](https://mlsanity.com/paper/2508.06810/) (2025)
- [Tibyan Corpus: Balanced and Comprehensive Error Coverage Corpus Using ChatGPT for Arabic Grammatical Error Correction](https://mlsanity.com/paper/2411.04588/) (2024)
- [RE2: Improving Chinese Grammatical Error Correction via Retrieving Appropriate Examples with Explanation](https://mlsanity.com/paper/2509.26038/) (2025)

## Abstract (arXiv)

In Grammatical Error Correction (GEC), it is crucial to ensure the user's comprehension of a reason for correction. Existing studies present tokens, examples, and hints as to the basis for correction but do not directly explain the reasons for corrections. Although methods that use Large Language Models (LLMs) to provide direct explanations in natural language have been proposed for various tasks, no such method exists for GEC. Generating explanations for GEC corrections involves aligning input and output tokens, identifying correction points, and presenting corresponding explanations consistently. However, it is not straightforward to specify a complex format to generate explanations, because explicit control of generation is difficult with prompts. This study introduces a method called controlled generation with Prompt Insertion (PI) so that LLMs can explain the reasons for corrections in natural language. In PI, LLMs first correct the input text, and then we automatically extract the correction points based on the rules. The extracted correction points are sequentially inserted into the LLM's explanation output as prompts, guiding the LLMs to generate explanations for the correction points. We also create an Explainable GEC (XGEC) dataset of correction reasons by annotating NUCLE, CoNLL2013, and CoNLL2014. Although generations from GPT-3 and ChatGPT using original prompts miss some correction points, the generation control using PI can explicitly guide to describe explanations for all correction points, contributing to improved performance in generating correction reasons.