# Chinese Spelling Correction as Rephrasing Language Model

arXiv: https://arxiv.org/abs/2308.08796 · HTML: https://mlsanity.com/paper/2308.08796/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 38 references that are in the map.
- First posted on arXiv 17 Aug 2023.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [GPT Understands, Too](https://mlsanity.com/paper/2103.10385/) (2021)
- [CLUE: A Chinese Language Understanding Evaluation Benchmark](https://mlsanity.com/paper/2004.05986/) (2020)
- [Better Fine-Tuning by Reducing Representational Collapse](https://mlsanity.com/paper/2008.03156/) (2020)
- [Spelling Error Correction with Soft-Masked BERT](https://mlsanity.com/paper/2005.07421/) (2020)
- [SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check](https://mlsanity.com/paper/2004.14166/) (2020)
- [General and Domain Adaptive Chinese Spelling Check with Error Consistent Pretraining](https://mlsanity.com/paper/2203.10929/) (2022)
- [Rethinking Masked Language Modeling for Chinese Spelling Correction](https://mlsanity.com/paper/2305.17721/) (2023)
- [Exploration and Exploitation: Two Ways to Improve Chinese Spelling Correction Models](https://mlsanity.com/paper/2105.14813/) (2021)
- [Adversarial Self-Attention for Language Understanding](https://mlsanity.com/paper/2206.12608/) (2022)
- [Spelling Correction as a Foreign Language](https://mlsanity.com/paper/1705.07371/) (2017)
- [Forging Multiple Training Objectives for Pre-trained Language Models via Meta-Learning](https://mlsanity.com/paper/2210.10293/) (2022)
- [Attack Named Entity Recognition by Entity Boundary Interference](https://mlsanity.com/paper/2305.05253/) (2023)

## Built on by (most cited first, 15 of 15)

- [CEC-Zero: Zero-Supervision Character Error Correction with Self-Generated Rewards](https://mlsanity.com/paper/2512.23971/) (2025)
- [A Simple yet Effective Training-free Prompt-free Approach to Chinese Spelling Correction Based on Large Language Models](https://mlsanity.com/paper/2410.04027/) (2024)
- [ASR-EC Benchmark: Evaluating Large Language Models on Chinese ASR Error Correction](https://mlsanity.com/paper/2412.03075/) (2024)
- [Mixture of Small and Large Models for Chinese Spelling Check](https://mlsanity.com/paper/2506.06887/) (2025)
- [Mitigating Catastrophic Forgetting in Multi-domain Chinese Spelling Correction by Multi-stage Knowledge Transfer Framework](https://mlsanity.com/paper/2402.11422/) (2024)
- [A Training-free LLM-based Approach to General Chinese Character Error Correction](https://mlsanity.com/paper/2502.15266/) (2025)
- [Enhancing Character-Level Understanding in LLMs through Token Internal Structure Learning](https://mlsanity.com/paper/2411.17679/) (2024)
- [Learning from Mistakes: Self-correct Adversarial Training for Chinese Unnatural Text Correction](https://mlsanity.com/paper/2412.17279/) (2024)
- [PERL: Pinyin Enhanced Rephrasing Language Model for Chinese ASR N-best Error Correction](https://mlsanity.com/paper/2412.03230/) (2024)
- [DISC: Plug-and-Play Decoding Intervention with Similarity of Characters for Chinese Spelling Check](https://mlsanity.com/paper/2412.12863/) (2024)
- [A Coin Has Two Sides: A Novel Detector-Corrector Framework for Chinese Spelling Correction](https://mlsanity.com/paper/2409.04150/) (2024)
- [CNMBERT: A Model for Converting Hanyu Pinyin Abbreviations to Chinese Characters](https://mlsanity.com/paper/2411.11770/) (2024)
- [LMSpell: Spell Correction with Pre-Trained Language Models](https://mlsanity.com/paper/2512.05414/) (2025)
- [CSRP: Chain-of-Thought Reasoning for Chinese Text Correction via Reinforcement Learning with Efficiency-Aware Rewards](https://mlsanity.com/paper/2606.00020/) (2026)
- [GUIDE: Generative Unsupervised Chinese Query Correction via Phonetic and Visual Shared-ID Encoding](https://mlsanity.com/paper/2608.25343/) (2026)

## Abstract (arXiv)

This paper studies Chinese Spelling Correction (CSC), which aims to detect and correct the potential spelling errors in a given sentence. Current state-of-the-art methods regard CSC as a sequence tagging task and fine-tune BERT-based models on sentence pairs. However, we note a critical flaw in the process of tagging one character to another, that the correction is excessively conditioned on the error. This is opposite from human mindset, where individuals rephrase the complete sentence based on its semantics, rather than solely on the error patterns memorized before. Such a counter-intuitive learning process results in the bottleneck of generalizability and transferability of machine spelling correction. To address this, we propose Rephrasing Language Model (ReLM), where the model is trained to rephrase the entire sentence by infilling additional slots, instead of character-to-character tagging. This novel training paradigm achieves the new state-of-the-art results across fine-tuned and zero-shot CSC benchmarks, outperforming previous counterparts by a large margin. Our method also learns transferable language representation when CSC is jointly trained with other tasks.