# General and Domain Adaptive Chinese Spelling Check with Error Consistent Pretraining

arXiv: https://arxiv.org/abs/2203.10929 · HTML: https://mlsanity.com/paper/2203.10929/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 40 references that are in the map.
- First posted on arXiv 21 Mar 2022.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Lexically Constrained Decoding for Sequence Generation Using Grid Beam Search](https://mlsanity.com/paper/1704.07138/) (2017)
- [Spelling Error Correction with Soft-Masked BERT](https://mlsanity.com/paper/2005.07421/) (2020)
- [Training Neural Machine Translation To Apply Terminology Constraints](https://mlsanity.com/paper/1906.01105/) (2019)
- [Automated Phrase Mining from Massive Text Corpora](https://mlsanity.com/paper/1702.04457/) (2017)
- [SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check](https://mlsanity.com/paper/2004.14166/) (2020)
- [The Past Mistake is the Future Wisdom: Error-driven Contrastive Probability Optimization for Chinese Spell Checking](https://mlsanity.com/paper/2203.00991/) (2022)
- [Code-Switching for Enhancing NMT with Pre-Specified Translation](https://mlsanity.com/paper/1904.09107/) (2019)
- [Read, Listen, and See: Leveraging Multimodal Information Helps Chinese Spell Checking](https://mlsanity.com/paper/2105.12306/) (2021)
- [Glyce: Glyph-vectors for Chinese Character Representations](https://mlsanity.com/paper/1901.10125/) (2019)
- [Lexically Constrained Neural Machine Translation with Levenshtein Transformer](https://mlsanity.com/paper/2004.12681/) (2020)
- [Neural Machine Translation Decoding with Terminology Constraints](https://mlsanity.com/paper/1805.03750/) (2018)
- [SYSTRAN's Pure Neural Machine Translation Systems](https://mlsanity.com/paper/1610.05540/) (2016)
- [Tail-to-Tail Non-Autoregressive Sequence Prediction for Chinese Grammatical Error Correction](https://mlsanity.com/paper/2106.01609/) (2021)
- [Exploration and Exploitation: Two Ways to Improve Chinese Spelling Correction Models](https://mlsanity.com/paper/2105.14813/) (2021)
- [Domain-shift Conditioning using Adaptable Filtering via Hierarchical Embeddings for Robust Chinese Spell Check](https://mlsanity.com/paper/2008.12281/) (2020)

## Built on by (most cited first, 25 of 25)

- [On the (In)Effectiveness of Large Language Models for Chinese Text Correction](https://mlsanity.com/paper/2307.09007/) (2023)
- [Towards Real-World Writing Assistance: A Chinese Character Checking Benchmark with Faked and Misspelled Characters](https://mlsanity.com/paper/2311.11268/) (2023)
- [Rethinking Masked Language Modeling for Chinese Spelling Correction](https://mlsanity.com/paper/2305.17721/) (2023)
- [Chinese Spelling Correction as Rephrasing Language Model](https://mlsanity.com/paper/2308.08796/) (2023)
- [CLongEval: A Chinese Benchmark for Evaluating Long-Context Large Language Models](https://mlsanity.com/paper/2403.03514/) (2024)
- [OpenBA: An Open-sourced 15B Bilingual Asymmetric seq2seq Model Pre-trained from Scratch](https://mlsanity.com/paper/2309.10706/) (2023)
- [Rich Semantic Knowledge Enhanced Large Language Models for Few-shot Chinese Spell Checking](https://mlsanity.com/paper/2403.08492/) (2024)
- [A Simple yet Effective Training-free Prompt-free Approach to Chinese Spelling Correction Based on Large Language Models](https://mlsanity.com/paper/2410.04027/) (2024)
- [HydraOpt: Navigating the Efficiency-Performance Trade-off of Adapter Merging](https://mlsanity.com/paper/2507.17706/) (2025)
- [An Error-Guided Correction Model for Chinese Spelling Error Correction](https://mlsanity.com/paper/2301.06323/) (2023)
- [Mitigating Catastrophic Forgetting in Multi-domain Chinese Spelling Correction by Multi-stage Knowledge Transfer Framework](https://mlsanity.com/paper/2402.11422/) (2024)
- [A Training-free LLM-based Approach to General Chinese Character Error Correction](https://mlsanity.com/paper/2502.15266/) (2025)
- [K-Merge: Online Continual Merging of Adapters for On-device Large Language Models](https://mlsanity.com/paper/2510.13537/) (2025)
- [CSED: A Chinese Semantic Error Diagnosis Corpus](https://mlsanity.com/paper/2305.05183/) (2023)
- [RSpell: Retrieval-augmented Framework for Domain Adaptive Chinese Spelling Check](https://mlsanity.com/paper/2308.08176/) (2023)
- [Chinese Spelling Correction: A Comprehensive Survey of Progress, Challenges, and Opportunities](https://mlsanity.com/paper/2502.11508/) (2025)
- [ChineseErrorCorrector3-4B: State-of-the-Art Chinese Spelling and Grammar Corrector](https://mlsanity.com/paper/2511.17562/) (2025)
- [PERL: Pinyin Enhanced Rephrasing Language Model for Chinese ASR N-best Error Correction](https://mlsanity.com/paper/2412.03230/) (2024)
- [Vision Language Models Are Not (Yet) Spelling Correctors](https://mlsanity.com/paper/2509.17418/) (2025)
- [An Empirical Investigation of Domain Adaptation Ability for Chinese Spelling Check Models](https://mlsanity.com/paper/2401.14630/) (2024)
- [A Coin Has Two Sides: A Novel Detector-Corrector Framework for Chinese Spelling Correction](https://mlsanity.com/paper/2409.04150/) (2024)
- [EdaCSC: Two Easy Data Augmentation Methods for Chinese Spelling Correction](https://mlsanity.com/paper/2409.05105/) (2024)
- [RAIR: Retrieval-Augmented Iterative Refinement for Chinese Spelling Correction](https://mlsanity.com/paper/2504.18938/) (2025)
- [Data-driven Clustering and Merging of Adapters for On-device Large Language Models](https://mlsanity.com/paper/2601.17441/) (2026)
- [CLFEC: A New Task for Unified Linguistic and Factual Error Correction in paragraph-level Chinese Professional Writing](https://mlsanity.com/paper/2602.23845/) (2026)

## Abstract (arXiv)

The lack of label data is one of the significant bottlenecks for Chinese Spelling Check (CSC). Existing researches use the method of automatic generation by exploiting unlabeled data to expand the supervised corpus. However, there is a big gap between the real input scenario and automatic generated corpus. Thus, we develop a competitive general speller ECSpell which adopts the Error Consistent masking strategy to create data for pretraining. This error consistency masking strategy is used to specify the error types of automatically generated sentences which is consistent with real scene. The experimental result indicates our model outperforms previous state-of-the-art models on the general benchmark. Moreover, spellers often work within a particular domain in real life. Due to lots of uncommon domain terms, experiments on our built domain specific datasets show that general models perform terribly. Inspired by the common practice of input methods, we propose to add an alterable user dictionary to handle the zero-shot domain adaption problem. Specifically, we attach a User Dictionary guided inference module (UD) to a general token classification based speller. Our experiments demonstrate that ECSpellUD, namely ECSpell combined with UD, surpasses all the other baselines largely, even approaching the performance on the general benchmark.