mlsanityData to 14 Oct 2026

To Err Is Human, but Llamas Can Learn It Too

arXiv 2403.05493 ↗First posted 8 Mar 2024cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

This study explores enhancing grammatical error correction (GEC) through artificial error generation (AEG) using language models (LMs). Specifically, we fine-tune Llama 2-based LMs for error generation and find that this approach yields synthetic errors akin to human errors. Next, we train GEC Llama models with the help of these artificial errors and outperform previous state-of-the-art error correction models, with gains ranging between 0.8 and 6 F0.5 points across all tested languages (German, Ukrainian, and Estonian). Moreover, we demonstrate that generating errors by fine-tuning smaller sequence-to-sequence models and prompting large commercial LMs (GPT-3.5 and GPT-4) also results in synthetic errors beneficially affecting error generation models.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 26 of its 65 references that are in the map. First posted on arXiv 8 Mar 2024.

Builds on

26 of 65 references.

References · 26

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  3. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  4. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  5. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  6. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  7. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
  8. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 548
  9. 2020GECToR -- Grammatical Error Correction: Tag, Not Rewritecited by 177
  10. 2021A Simple Recipe for Multilingual Grammatical Error Correctioncited by 108
  11. 2023Is ChatGPT a Highly Fluent Grammatical Error Correction System? A Comprehensive Evaluationcited by 97
  12. 2023ChatGPT or Grammarly? Evaluating ChatGPT on Grammatical Error Correction Benchmarkcited by 77
14 more
  1. 2020Seq2Edits: Sequence Transduction Using Span-level Edit Operationscited by 58
  2. 2018Approaching Neural Grammatical Error Correction as a Low-Resource Machine Translation Taskcited by 56
  3. 2021Synthetic Data Generation for Grammatical Error Correction with Tagged Corruption Modelscited by 56
  4. 2019An Empirical Study of Incorporating Pseudo Data into Grammatical Error Correctioncited by 55
  5. 2019Improving Grammatical Error Correction via Pre-Training a Copy-Augmented Architecture with Unlabeled Datacited by 50
  6. 2023Multi-Task Instruction Tuning of LLaMa for Specific Scenarios: A Preliminary Study on Writing Assistancecited by 33
  7. 2019Grammatical Error Correction in Low-Resource Scenarioscited by 28
  8. 2021UA-GEC: Grammatical Error Correction and Fluency Corpus for the Ukrainian Languagecited by 19
  9. 2022Czech Grammar Error Correction with a Large and Diverse Corpuscited by 17
  10. 2017Artificial Error Generation with Machine Translation and Syntactic Patternscited by 11
  11. 2018Wronging a Right: Generating Better Errors to Improve Grammatical Error Detectioncited by 11
  12. 2019The Unbearable Weight of Generating Artificial Errors for Grammatical Error Correctioncited by 8
  13. 2023Byte-Level Grammatical Error Correction Using Synthetic and Curated Corporacited by 7
  14. 2023GECTurk: Grammatical Error Correction and Detection Dataset for Turkishcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2026Machine-Assisted Grading of Nationwide School-Leaving Essay Exams with LLMs and Statistical NLPcited by 2
  2. 2025Data Augmentation for Spoken Grammatical Error Correctioncited by 1
  3. 2024Zero-shot Cross-Lingual Transfer for Synthetic Data Generation in Grammatical Error Detectioncited by 0