mlsanityData to 14 Oct 2026

Chinese grammatical error correction based on knowledge distillation

arXiv 2208.00351 ↗First posted 31 Jul 2022cs.CL cs.AIcited by 1

On the map: Knowledge distillation → Knowledge distillation for LLMs

Abstract (arXiv)

In view of the poor robustness of existing Chinese grammatical error correction models on attack test sets and large model parameters, this paper uses the method of knowledge distillation to compress model parameters and improve the anti-attack ability of the model. In terms of data, the attack test set is constructed by integrating the disturbance into the standard evaluation data set, and the model robustness is evaluated by the attack test set. The experimental results show that the distilled small model can ensure the performance and improve the training speed under the condition of reducing the number of model parameters, and achieve the optimal effect on the attack test set, and the robustness is significantly improved. Code is available at https://github.com/Richard88888/KD-CGEC.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Knowledge distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 12 of its 37 references that are in the map. First posted on arXiv 31 Jul 2022.

Builds on

12 of 37 references.

References · 12

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2016Layer Normalizationcited by 5,669
  5. 2016Sequence-Level Knowledge Distillationcited by 801
  6. 2020Self-Knowledge Distillation with Progressive Refinement of Targetscited by 107
  7. 2017JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correctioncited by 97
  8. 2017Visual Relationship Detection with Internal and External Linguistic Knowledge Distillationcited by 61
  9. 2018A Multilayer Convolutional Encoder-Decoder Neural Network for Grammatical Error Correctioncited by 36
  10. 2021Distill on the Go: Online knowledge distillation in self-supervised learningcited by 22
  11. 2017A Nested Attention Neural Hybrid Model for Grammatical Error Correctioncited by 12
  12. 2020Ensemble Distillation Approaches for Grammatical Error Correctioncited by 8

Built on by

Cited by year

Citing papers per year: 2023: 1.

1 paper.

  1. 2023Grammatical Error Correction via Mixed-Grained Weighted Trainingcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2023An Error-Guided Correction Model for Chinese Spelling Error Correction
  2. 2024Rethinking the Roles of Large Language Models in Chinese Grammatical Error Correction
  3. 2024Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Application
  4. 2025Contrastive Knowledge Transfer and Robust Optimization for Secure Alignment of Large Language Models
  5. 2022Hard Gate Knowledge Distillation -- Leverage Calibration for Robust and Reliable Language Model
  6. 2023PaD: Program-aided Distillation Can Teach Small Models Reasoning Better than Chain-of-thought Fine-tuning
  7. 2025CEC-Zero: Chinese Error Correction Solution Based on LLM
  8. 2026CSRP: Chain-of-Thought Reasoning for Chinese Text Correction via Reinforcement Learning with Efficiency-Aware Rewards