# GrammarGPT: Exploring Open-Source LLMs for Native Chinese Grammatical Error Correction with Supervised Fine-Tuning

arXiv: https://arxiv.org/abs/2307.13923 · HTML: https://mlsanity.com/paper/2307.13923/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 29 references that are in the map.
- First posted on arXiv 26 Jul 2023.

## Builds on (most cited first, 18 of 18)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [Instruction Tuning with GPT-4](https://mlsanity.com/paper/2304.03277/) (2023)
- [Large Language Models Are Reasoning Teachers](https://mlsanity.com/paper/2212.10071/) (2022)
- [HuatuoGPT, towards Taming Language Model to Be a Doctor](https://mlsanity.com/paper/2305.15075/) (2023)
- [InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction](https://mlsanity.com/paper/2304.08085/) (2023)
- [A Simple Recipe for Multilingual Grammatical Error Correction](https://mlsanity.com/paper/2106.03830/) (2021)
- [SCOTT: Self-Consistent Chain-of-Thought Distillation](https://mlsanity.com/paper/2305.01879/) (2023)
- [Is ChatGPT a Highly Fluent Grammatical Error Correction System? A Comprehensive Evaluation](https://mlsanity.com/paper/2304.01746/) (2023)
- [Instruct-FinGPT: Financial Sentiment Analysis by Instruction Tuning of General-Purpose Large Language Models](https://mlsanity.com/paper/2306.12659/) (2023)
- [Stronger Baselines for Grammatical Error Correction Using Pretrained Encoder-Decoder Model](https://mlsanity.com/paper/2005.11849/) (2020)
- [MuCGEC: a Multi-Reference Multi-Source Evaluation Dataset for Chinese Grammatical Error Correction](https://mlsanity.com/paper/2204.10994/) (2022)
- [Phoenix: Democratizing ChatGPT across Languages](https://mlsanity.com/paper/2304.10453/) (2023)
- [Linguistic Rules-Based Corpus Generation for Native Chinese Grammatical Error Correction](https://mlsanity.com/paper/2210.10442/) (2022)
- [Sequence-to-Action: Grammatical Error Correction with Action Guided Sequence Generation](https://mlsanity.com/paper/2205.10884/) (2022)
- [NaSGEC: a Multi-Domain Chinese Grammatical Error Correction Dataset from Native Speaker Texts](https://mlsanity.com/paper/2305.16023/) (2023)

## Built on by (most cited first, 23 of 23)

- [Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity](https://mlsanity.com/paper/2310.07521/) (2023)
- [Large Language Models for Education: A Survey and Outlook](https://mlsanity.com/paper/2403.18105/) (2024)
- [Large Language Models for Education: A Survey](https://mlsanity.com/paper/2405.13001/) (2024)
- [Who Wrote This? The Key to Zero-Shot LLM-Generated Text Detection Is GECScore](https://mlsanity.com/paper/2405.04286/) (2024)
- [From Google Gemini to OpenAI Q* (Q-Star): A Survey of Reshaping the Generative Artificial Intelligence (AI) Research Landscape](https://mlsanity.com/paper/2312.10868/) (2023)
- [Adapting Large Language Models for Education: Foundational Capabilities, Potentials, and Challenges](https://mlsanity.com/paper/2401.08664/) (2023)
- [Enhancing Emergency Decision-making with Knowledge Graphs and Large Language Models](https://mlsanity.com/paper/2311.08732/) (2023)
- [Rethinking the Roles of Large Language Models in Chinese Grammatical Error Correction](https://mlsanity.com/paper/2402.11420/) (2024)
- [Improving Grammatical Error Correction via Contextual Data Augmentation](https://mlsanity.com/paper/2406.17456/) (2024)
- [Alirector: Alignment-Enhanced Chinese Grammatical Error Corrector](https://mlsanity.com/paper/2402.04601/) (2024)
- [Towards Self-Robust LLMs: Intrinsic Prompt Noise Resistance via CoIPO](https://mlsanity.com/paper/2603.03314/) (2026)
- [Neural Spelling: A Spell-Based BCI System for Language Neural Decoding](https://mlsanity.com/paper/2501.17489/) (2025)
- [MedErrBench: A Fine-Grained Multilingual Benchmark for Medical Error Detection and Correction with Clinical Expert Annotations](https://mlsanity.com/paper/2602.05692/) (2026)
- [DKDL-Net: A Lightweight Bearing Fault Detection Model via Decoupled Knowledge Distillation and Low-Rank Adaptation Fine-tuning](https://mlsanity.com/paper/2406.06653/) (2024)
- [Tibyan Corpus: Balanced and Comprehensive Error Coverage Corpus Using ChatGPT for Arabic Grammatical Error Correction](https://mlsanity.com/paper/2411.04588/) (2024)
- [Explanation based In-Context Demonstrations Retrieval for Multilingual Grammatical Error Correction](https://mlsanity.com/paper/2502.08507/) (2025)
- [An Analysis of Language Frequency and Error Correction for Esperanto](https://mlsanity.com/paper/2402.09696/) (2024)
- [A Simple Yet Effective Corpus Construction Framework for Indonesian Grammatical Error Correction](https://mlsanity.com/paper/2410.20838/) (2024)
- [Hide and Seek with LLMs: An Adversarial Game for Sneaky Error Generation and Self-Improving Diagnosis](https://mlsanity.com/paper/2508.03396/) (2025)
- [RE2: Improving Chinese Grammatical Error Correction via Retrieving Appropriate Examples with Explanation](https://mlsanity.com/paper/2509.26038/) (2025)
- [Sandwich Reasoning: An Answer-Reasoning-Answer Approach for Low-Latency Query Correction](https://mlsanity.com/paper/2601.03672/) (2026)
- [Encode Errors: Representational Retrieval of In-Context Demonstrations for Multilingual Grammatical Error Correction](https://mlsanity.com/paper/2606.15416/) (2026)
- [IRWOZ 2.0: A Large Language Model-driven Dialogue Dataset for Industrial Robot Conversations](https://mlsanity.com/paper/2609.04030/) (2026)

## Abstract (arXiv)

Grammatical error correction aims to correct ungrammatical sentences automatically. Recently, some work has demonstrated the excellent capabilities of closed-source Large Language Models (LLMs, e.g., ChatGPT) in grammatical error correction. However, the potential of open-source LLMs remains unexplored. In this paper, we introduced GrammarGPT, an open-source LLM, to preliminary explore its potential for native Chinese grammatical error correction. The core recipe of GrammarGPT is to leverage the hybrid dataset of ChatGPT-generated and human-annotated. For grammatical errors with clues, we proposed a heuristic method to guide ChatGPT to generate ungrammatical sentences by providing those clues. For grammatical errors without clues, we collected ungrammatical sentences from publicly available websites and manually corrected them. In addition, we employed an error-invariant augmentation method to enhance the ability of the model to correct native Chinese grammatical errors. We ultimately constructed about 1k parallel data and utilized these data to fine-tune open-source LLMs (e.g., Phoenix, released by The Chinese University of Hong Kong, Shenzhen) with instruction tuning. The experimental results show that GrammarGPT outperforms the existing SOTA system significantly. Although model parameters are 20x larger than the SOTA baseline, the required amount of data for instruction tuning is 1200x smaller, illustrating the potential of open-source LLMs on native CGEC. Our GrammarGPT ranks 3rd on NLPCC2023 SharedTask1, demonstrating our approach's effectiveness. The code and data are available at urlhttps://github.com/FreedomIntelligence/GrammarGPT.