mlsanityData to 14 Oct 2026

Zero-shot Cross-Lingual Transfer for Synthetic Data Generation in Grammatical Error Detection

arXiv 2407.11854 ↗First posted 16 Jul 2024cs.CL cs.AIcited by 0

On the map: Synthetic data and self-improvement for LLMs → LLM synthetic data generation

Abstract (arXiv)

Grammatical Error Detection (GED) methods rely heavily on human annotated error corpora. However, these annotations are unavailable in many low-resource languages. In this paper, we investigate GED in this context. Leveraging the zero-shot cross-lingual transfer capabilities of multilingual pre-trained language models, we train a model using data from a diverse set of languages to generate synthetic errors in other languages. These synthetic error corpora are then used to train a GED model. Specifically we propose a two-stage fine-tuning pipeline where the GED model is first fine-tuned on multilingual synthetic data from target languages followed by fine-tuning on human-annotated GED corpora from source languages. This approach outperforms current state-of-the-art annotation-free GED methods. We also analyse the errors produced by our method and other strong baselines, finding that our approach produces errors that are more diverse and more similar to human errors.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 32 of its 68 references that are in the map. First posted on arXiv 16 Jul 2024.

Builds on

32 of 68 references.

References · 32

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  4. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  5. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
  6. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
  7. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 723
  8. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 422
  9. 2020InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Trainingcited by 237
  10. 2021A Simple Recipe for Multilingual Grammatical Error Correctioncited by 108
  11. 2023Is ChatGPT a Highly Fluent Grammatical Error Correction System? A Comprehensive Evaluationcited by 97
  12. 2019Parallel Iterative Edit Models for Local Sequence Transductioncited by 81
20 more
  1. 2023ChatGPT or Grammarly? Evaluating ChatGPT on Grammatical Error Correction Benchmarkcited by 77
  2. 2024Multilingual Instruction Tuning With Just a Pinch of Multilingualitycited by 73
  3. 2021Synthetic Data Generation for Grammatical Error Correction with Tagged Corruption Modelscited by 56
  4. 2019An Empirical Study of Incorporating Pseudo Data into Grammatical Error Correctioncited by 55
  5. 2019Corpora Generation for Grammatical Error Correctioncited by 53
  6. 2021LM-Critic: Language Models for Unsupervised Grammatical Error Correctioncited by 33
  7. 2024Reuse Your Rewards: Reward Model Transfer for Zero-Shot Cross-Lingual Alignmentcited by 26
  8. 2024Prompting open-source and commercial language models for grammatical error correction of English learner textcited by 24
  9. 2023Advancements in Arabic Grammatical Error Detection and Correction: An Empirical Investigationcited by 22
  10. 2021Automatic Error Type Annotation for Arabiccited by 21
  11. 2023Enhancing Grammatical Error Correction Systems with Explanationscited by 19
  12. 2022Czech Grammar Error Correction with a Large and Diverse Corpuscited by 17
  13. 2016Compositional Sequence Labeling Models for Error Detection in Learner Writingcited by 16
  14. 2022A Unified Strategy for Multilingual Grammatical Error Correction with Pre-trained Cross-Lingual Language Modelcited by 16
  15. 2019Context is Key: Grammatical Error Detection with Contextual Word Representationscited by 14
  16. 2018Wronging a Right: Generating Better Errors to Improve Grammatical Error Detectioncited by 11
  17. 2024Key ingredients for effective zero-shot cross-lingual knowledge transfer in generative taskscited by 11
  18. 2023Improving Seq2Seq Grammatical Error Correction via Decoding Interventionscited by 8
  19. 2019Multi-Head Multi-Layer Attention to Deep Language Representations for Grammatical Error Detectioncited by 7
  20. 2024To Err Is Human, but Llamas Can Learn It Toocited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024ChatLang-8: An LLM-Based Synthetic Data Generation Framework for Grammatical Error Correction
  2. 2025BhashaKritika: Building Synthetic Pretraining Data at Scale for Indic Languages
  3. 2024Leveraging LLMs for Bangla Grammar Error Correction:Error Categorization, Synthetic Data, and Model Evaluation
  4. 2025ZeroED: Hybrid Zero-shot Error Detection through Large Language Model Reasoning
  5. 2026SynthSentry: Detecting Synthetic Data Contamination in Language Model Training Data
  6. 2025Artificial Conversations, Real Results: Fostering Language Detection with Synthetic Data
  7. 2026A Factorial Study of Synthetic Data Generation for Low-Resource Machine Translation using Grammar Books
  8. 2026Beyond Reference-Based Evaluation: Reward Models for Meta-Evaluation of Grammatical Error Correction