# Chinese grammatical error correction based on knowledge distillation

arXiv: https://arxiv.org/abs/2208.00351 · HTML: https://mlsanity.com/paper/2208.00351/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge distillation for LLMs, one of 12 directions in Knowledge distillation.
- Builds on 12 of its 37 references that are in the map.
- First posted on arXiv 31 Jul 2022.

## Placed in

- Knowledge distillation / [Knowledge distillation for LLMs](https://mlsanity.com/distill/knowledge-distillation-for-llms/)

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Sequence-Level Knowledge Distillation](https://mlsanity.com/paper/1606.07947/) (2016)
- [Self-Knowledge Distillation with Progressive Refinement of Targets](https://mlsanity.com/paper/2006.12000/) (2020)
- [JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correction](https://mlsanity.com/paper/1702.04066/) (2017)
- [Visual Relationship Detection with Internal and External Linguistic Knowledge Distillation](https://mlsanity.com/paper/1707.09423/) (2017)
- [A Multilayer Convolutional Encoder-Decoder Neural Network for Grammatical Error Correction](https://mlsanity.com/paper/1801.08831/) (2018)
- [Distill on the Go: Online knowledge distillation in self-supervised learning](https://mlsanity.com/paper/2104.09866/) (2021)
- [A Nested Attention Neural Hybrid Model for Grammatical Error Correction](https://mlsanity.com/paper/1707.02026/) (2017)
- [Ensemble Distillation Approaches for Grammatical Error Correction](https://mlsanity.com/paper/2012.07535/) (2020)

## Built on by (most cited first, 1 of 1)

- [Grammatical Error Correction via Mixed-Grained Weighted Training](https://mlsanity.com/paper/2311.13848/) (2023)

## Abstract (arXiv)

In view of the poor robustness of existing Chinese grammatical error correction models on attack test sets and large model parameters, this paper uses the method of knowledge distillation to compress model parameters and improve the anti-attack ability of the model. In terms of data, the attack test set is constructed by integrating the disturbance into the standard evaluation data set, and the model robustness is evaluated by the attack test set. The experimental results show that the distilled small model can ensure the performance and improve the training speed under the condition of reducing the number of model parameters, and achieve the optimal effect on the attack test set, and the robustness is significantly improved. Code is available at https://github.com/Richard88888/KD-CGEC.