# Exploration and Exploitation: Two Ways to Improve Chinese Spelling Correction Models

arXiv: https://arxiv.org/abs/2105.14813 · HTML: https://mlsanity.com/paper/2105.14813/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 21 references that are in the map.
- First posted on arXiv 31 May 2021.

## Builds on (most cited first, 8 of 8)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Adversarial Examples for Evaluating Reading Comprehension Systems](https://mlsanity.com/paper/1707.07328/) (2017)
- [Generating Natural Adversarial Examples](https://mlsanity.com/paper/1710.11342/) (2017)
- [Neural Responding Machine for Short-Text Conversation](https://mlsanity.com/paper/1503.02364/) (2015)
- [Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples](https://mlsanity.com/paper/1803.01128/) (2018)
- [Spelling Error Correction with Soft-Masked BERT](https://mlsanity.com/paper/2005.07421/) (2020)
- [SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check](https://mlsanity.com/paper/2004.14166/) (2020)

## Built on by (most cited first, 17 of 17)

- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [The Past Mistake is the Future Wisdom: Error-driven Contrastive Probability Optimization for Chinese Spell Checking](https://mlsanity.com/paper/2203.00991/) (2022)
- [On the (In)Effectiveness of Large Language Models for Chinese Text Correction](https://mlsanity.com/paper/2307.09007/) (2023)
- [Learning from the Dictionary: Heterogeneous Knowledge Guided Fine-tuning for Chinese Spell Checking](https://mlsanity.com/paper/2210.10320/) (2022)
- [General and Domain Adaptive Chinese Spelling Check with Error Consistent Pretraining](https://mlsanity.com/paper/2203.10929/) (2022)
- [Rethinking Masked Language Modeling for Chinese Spelling Correction](https://mlsanity.com/paper/2305.17721/) (2023)
- [Chinese Spelling Correction as Rephrasing Language Model](https://mlsanity.com/paper/2308.08796/) (2023)
- [A Frustratingly Easy Plug-and-Play Detection-and-Reasoning Module for Chinese Spelling Check](https://mlsanity.com/paper/2310.09119/) (2023)
- [Correct Like Humans: Progressive Learning Framework for Chinese Text Error Correction](https://mlsanity.com/paper/2306.17447/) (2023)
- [uChecker: Masked Pretrained Language Models as Unsupervised Chinese Spelling Checkers](https://mlsanity.com/paper/2209.07068/) (2022)
- [MCSCSet: A Specialist-annotated Dataset for Medical-domain Chinese Spelling Correction](https://mlsanity.com/paper/2210.11720/) (2022)
- [SDCL: Self-Distillation Contrastive Learning for Chinese Spell Checking](https://mlsanity.com/paper/2210.17168/) (2022)
- [Pretraining Chinese BERT for Detecting Word Insertion and Deletion Errors](https://mlsanity.com/paper/2204.12052/) (2022)
- [Block the Label and Noise: An N-Gram Masked Speller for Chinese Spell Checking](https://mlsanity.com/paper/2305.03314/) (2023)
- [Make BERT-based Chinese Spelling Check Model Enhanced by Layerwise Attention and Gaussian Mixture Model](https://mlsanity.com/paper/2312.16623/) (2023)
- [EdaCSC: Two Easy Data Augmentation Methods for Chinese Spelling Correction](https://mlsanity.com/paper/2409.05105/) (2024)
- [Adaptive Sparse Softmax: An Effective and Efficient Softmax Variant](https://mlsanity.com/paper/2508.03175/) (2025)

## Abstract (arXiv)

A sequence-to-sequence learning with neural networks has empirically proven to be an effective framework for Chinese Spelling Correction (CSC), which takes a sentence with some spelling errors as input and outputs the corrected one. However, CSC models may fail to correct spelling errors covered by the confusion sets, and also will encounter unseen ones. We propose a method, which continually identifies the weak spots of a model to generate more valuable training instances, and apply a task-specific pre-training strategy to enhance the model. The generated adversarial examples are gradually added to the training set. Experimental results show that such an adversarial training method combined with the pretraining strategy can improve both the generalization and robustness of multiple CSC models across three different datasets, achieving stateof-the-art performance for CSC task.