mlsanityData to 7 Oct 2026

Is ChatGPT a Highly Fluent Grammatical Error Correction System? A Comprehensive Evaluation

arXiv 2304.01746 ↗First posted 4 Apr 2023cs.CLcited by 97

Not on the map. Why?

Abstract (arXiv)

ChatGPT, a large-scale language model based on the advanced GPT-3.5 architecture, has shown remarkable potential in various Natural Language Processing (NLP) tasks. However, there is currently a dearth of comprehensive study exploring its potential in the area of Grammatical Error Correction (GEC). To showcase its capabilities in GEC, we design zero-shot chain-of-thought (CoT) and few-shot CoT settings using in-context learning for ChatGPT. Our evaluation involves assessing ChatGPT's performance on five official test sets in three different languages, along with three document-level GEC test sets in English. Our experimental results and human evaluations demonstrate that ChatGPT has excellent error detection capabilities and can freely correct errors to make the corrected sentences very fluent, possibly due to its over-correction tendencies and not adhering to the principle of minimal edits. Additionally, its performance in non-English and low-resource settings highlights its potential in multilingual GEC tasks. However, further analysis of various types of errors at the document-level has shown that ChatGPT cannot effectively correct agreement, coreference, tense errors across sentences, and cross-sentence boundary errors.

Key facts. Cited by 97 papers in the map (data through 7 Oct 2026). Builds on 19 of its 34 references that are in the map. First posted on arXiv 4 Apr 2023.

Builds on

19 of 34 references.

References · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  5. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  6. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 486
  7. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
  8. 2020GECToR -- Grammatical Error Correction: Tag, Not Rewritecited by 177
  9. 2023Is ChatGPT A Good Translator? Yes With GPT-4 As The Enginecited by 172
  10. 2022Prompting PaLM for Translation: Assessing Strategies and Performancecited by 144
  11. 2023Towards Making the Most of ChatGPT for Machine Translationcited by 138
  12. 2023Exploring the Limits of ChatGPT for Query or Aspect-based Text Summarizationcited by 122
7 more
  1. 2023A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capabilitycited by 118
  2. 2021A Simple Recipe for Multilingual Grammatical Error Correctioncited by 108
  3. 2017JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correctioncited by 96
  4. 2023ChatGPT or Grammarly? Evaluating ChatGPT on Grammatical Error Correction Benchmarkcited by 77
  5. 2021Synthetic Data Generation for Grammatical Error Correction with Tagged Corruption Modelscited by 56
  6. 2020Stronger Baselines for Grammatical Error Correction Using Pretrained Encoder-Decoder Modelcited by 33
  7. 2016GLEU Without Tuningcited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 32; 2024: 40; 2025: 18; 2026: 7.

97 papers.

Sort:

By area:

  1. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  2. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  3. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 224
  4. 2023ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learningcited by 203
  5. 2023Alleviating Hallucinations of Large Language Models through Induced Hallucinationscited by 78
  6. 2023CoEdIT: Text Editing by Task-Specific Instruction Tuningcited by 67
  7. 2024MoRAL: MoE Augmented LoRA for LLMs' Lifelong Learningcited by 63
  8. 2023Can Generative Large Language Models Perform ASR Error Correction?cited by 56
  9. 2023On the (In)Effectiveness of Large Language Models for Chinese Text Correctioncited by 53
  10. 2023Evaluation Metrics in the Era of GPT-4: Reliably Evaluating Large Language Models on Sequence to Sequence Taskscited by 53
  11. 2023Exploring Effectiveness of GPT-3 in Grammatical Error Correction: A Study on Performance and Controllability in Prompt-Based Methodscited by 37
  12. 2024DetectRL: Benchmarking LLM-Generated Text Detection in Real-World Scenarioscited by 37
38 more
  1. 2023Multi-Task Instruction Tuning of LLaMa for Specific Scenarios: A Preliminary Study on Writing Assistancecited by 33
  2. 2024Who Wrote This? The Key to Zero-Shot LLM-Generated Text Detection Is GECScorecited by 27
  3. 2024Prompting open-source and commercial language models for grammatical error correction of English learner textcited by 24
  4. 2023GrammarGPT: Exploring Open-Source LLMs for Native Chinese Grammatical Error Correction with Supervised Fine-Tuningcited by 23
  5. 2023RELIC: Investigating Large Language Model Responses using Self-Consistencycited by 23
  6. 2024ASR Error Correction using Large Language Modelscited by 23
  7. 2023Advancements in Arabic Grammatical Error Detection and Correction: An Empirical Investigationcited by 22
  8. 2024GPT-3.5 for Grammatical Error Correctioncited by 18
  9. 2023Specialist or Generalist? Instruction Tuning for Specific NLP Taskscited by 17
  10. 2024Salute the Classic: Revisiting Challenges of Machine Translation in the Age of Large Language Modelscited by 17
  11. 2023Correct Like Humans: Progressive Learning Framework for Chinese Text Error Correctioncited by 13
  12. 2023Adapting Large Language Models for Education: Foundational Capabilities, Potentials, and Challengescited by 13
  13. 2024mEdIT: Multilingual Text Editing via Instruction Tuningcited by 13
  14. 2023Beyond English: Evaluating LLMs for Arabic Grammatical Error Correctioncited by 12
  15. 2024Anchor-based Large Language Modelscited by 12
  16. 2024C-LLM: Learn to Check Chinese Spelling Errors Character by Charactercited by 12
  17. 2023How Far Are LLMs from Believable AI? A Benchmark for Evaluating the Believability of Human Behavior Simulationcited by 11
  18. 2024Robustness of Large Language Models to Perturbations in Textcited by 11
  19. 2024mCSQA: Multilingual Commonsense Reasoning Dataset with Unified Creation Strategy by Language Models and Humanscited by 10
  20. 2025QE-RAG: A Robust Retrieval-Augmented Generation Benchmark for Query Entry Errorscited by 10
  21. 2025Benchmarking the Detection of LLMs-Generated Modern Chinese Poetrycited by 10
  22. 2023NaSGEC: a Multi-Domain Chinese Grammatical Error Correction Dataset from Native Speaker Textscited by 9
  23. 2024Rethinking the Roles of Large Language Models in Chinese Grammatical Error Correctioncited by 9
  24. 2024Evaluating Prompting Strategies for Grammatical Error Correction Based on Language Proficiencycited by 9
  25. 2025Enhancing Text Editing for Grammatical Error Correction: Arabic as a Case Studycited by 9
  26. 2023Non-autoregressive Text Editing with Copy-aware Latent Alignmentscited by 8
  27. 2024Pillars of Grammatical Error Correction: Comprehensive Inspection Of Contemporary Approaches In The Era of Large Language Modelscited by 8
  28. 2023Controlled Generation with Prompt Insertion for Natural Language Explanations in Grammatical Error Correctioncited by 7
  29. 2024LLMs in Education: Novel Perspectives, Challenges, and Opportunitiescited by 7
  30. 2023RobustGEC: Robust Grammatical Error Correction Against Subtle Context Perturbationcited by 6
  31. 2024Improving Grammatical Error Correction via Contextual Data Augmentationcited by 6
  32. 2025Position: LLMs Can be Good Tutors in English Educationcited by 6
  33. 2025Opportunities and Challenges of LLMs in Education: An NLP Perspectivecited by 6
  34. 2023Reducing Sequence Length by Predicting Edit Operations with Large Language Modelscited by 5
  35. 2023Evaluating GPT-3.5 and GPT-4 on Grammatical Error Correction for Brazilian Portuguesecited by 5
  36. 2023Evaluating the Capability of Large-scale Language Models on Chinese Grammatical Error Correction Taskcited by 5
  37. 2023ChatGPT for Arabic Grammatical Error Correctioncited by 5
  38. 2023Efficient Grammatical Error Correction Via Multi-Task Training and Optimized Training Schedulecited by 5

50 most cited of 97 All 97 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models
  2. 2023A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets
  3. 2023Pushing the Limits of ChatGPT on NLP Tasks
  4. 2023Is ChatGPT a Good Causal Reasoner? A Comprehensive Evaluation
  5. 2023GPT-3.5, GPT-4, or BARD? Evaluating LLMs Reasoning Ability in Zero-Shot Setting and Performance Boosting Through Prompts
  6. 2026Beyond Reference-Based Evaluation: Reward Models for Meta-Evaluation of Grammatical Error Correction
  7. 2024Leveraging LLMs for Bangla Grammar Error Correction:Error Categorization, Synthetic Data, and Model Evaluation
  8. 2023A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models