# Refining Corpora from a Model Calibration Perspective for Chinese Spelling Correction

arXiv: https://arxiv.org/abs/2407.15498 · HTML: https://mlsanity.com/paper/2407.15498/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 38 references that are in the map.
- First posted on arXiv 22 Jul 2024.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [Confident Learning: Estimating Uncertainty in Dataset Labels](https://mlsanity.com/paper/1911.00068/) (2019)
- [SELF: Learning to Filter Noisy Labels with Self-Ensembling](https://mlsanity.com/paper/1910.01842/) (2019)
- [Self-Adaptive Training: beyond Empirical Risk Minimization](https://mlsanity.com/paper/2002.10319/) (2020)
- [Spelling Error Correction with Soft-Masked BERT](https://mlsanity.com/paper/2005.07421/) (2020)
- [SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check](https://mlsanity.com/paper/2004.14166/) (2020)
- [The Past Mistake is the Future Wisdom: Error-driven Contrastive Probability Optimization for Chinese Spell Checking](https://mlsanity.com/paper/2203.00991/) (2022)
- [Read, Listen, and See: Leveraging Multimodal Information Helps Chinese Spell Checking](https://mlsanity.com/paper/2105.12306/) (2021)
- [Learning from the Dictionary: Heterogeneous Knowledge Guided Fine-tuning for Chinese Spell Checking](https://mlsanity.com/paper/2210.10320/) (2022)
- [Improving Chinese Spelling Check by Character Pronunciation Prediction: The Effects of Adaptivity and Granularity](https://mlsanity.com/paper/2210.10996/) (2022)
- [Disentangled Phonetic Representation for Chinese Spelling Correction](https://mlsanity.com/paper/2305.14783/) (2023)
- [Domain-shift Conditioning using Adaptable Filtering via Hierarchical Embeddings for Robust Chinese Spell Check](https://mlsanity.com/paper/2008.12281/) (2020)

## Built on by (most cited first, 1 of 1)

- [Unveiling the Impact of Multimodal Features on Chinese Spelling Correction: From Analysis to Design](https://mlsanity.com/paper/2504.07661/) (2025)

## Abstract (arXiv)

Chinese Spelling Correction (CSC) commonly lacks large-scale high-quality corpora, due to the labor-intensive labeling of spelling errors in real-life human writing or typing scenarios. Two data augmentation methods are widely adopted: (1) textitRandom Replacement with the guidance of confusion sets and (2) textitOCR/ASR-based Generation that simulates character misusing. However, both methods inevitably introduce noisy data (e.g., false spelling errors), potentially leading to over-correction. By carefully analyzing the two types of corpora, we find that though the latter achieves more robust generalization performance, the former yields better-calibrated CSC models. We then provide a theoretical analysis of this empirical observation, based on which a corpus refining strategy is proposed. Specifically, OCR/ASR-based data samples are fed into a well-calibrated CSC model trained on random replacement-based corpora and then filtered based on prediction confidence. By learning a simple BERT-based model on the refined OCR/ASR-based corpus, we set up impressive state-of-the-art performance on three widely-used benchmarks, while significantly alleviating over-correction (e.g., lowering false positive predictions).