# Learning from the Dictionary: Heterogeneous Knowledge Guided Fine-tuning for Chinese Spell Checking

arXiv: https://arxiv.org/abs/2210.10320 · HTML: https://mlsanity.com/paper/2210.10320/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 39 references that are in the map.
- First posted on arXiv 19 Oct 2022.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Revisiting Pre-Trained Models for Chinese Natural Language Processing](https://mlsanity.com/paper/2004.13922/) (2020)
- [Gate-Variants of Gated Recurrent Unit (GRU) Neural Networks](https://mlsanity.com/paper/1701.05923/) (2017)
- [ERNIE 2.0: A Continual Pre-training Framework for Language Understanding](https://mlsanity.com/paper/1907.12412/) (2019)
- [Self-Guided Contrastive Learning for BERT Sentence Representations](https://mlsanity.com/paper/2106.07345/) (2021)
- [Spelling Error Correction with Soft-Masked BERT](https://mlsanity.com/paper/2005.07421/) (2020)
- [ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning](https://mlsanity.com/paper/2012.15022/) (2020)
- [SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check](https://mlsanity.com/paper/2004.14166/) (2020)
- [The Past Mistake is the Future Wisdom: Error-driven Contrastive Probability Optimization for Chinese Spell Checking](https://mlsanity.com/paper/2203.00991/) (2022)
- [Read, Listen, and See: Leveraging Multimodal Information Helps Chinese Spell Checking](https://mlsanity.com/paper/2105.12306/) (2021)
- [Contrastive Learning with Hard Negative Entities for Entity Set Expansion](https://mlsanity.com/paper/2204.07789/) (2022)
- [Exploration and Exploitation: Two Ways to Improve Chinese Spelling Correction Models](https://mlsanity.com/paper/2105.14813/) (2021)

## Built on by (most cited first, 30 of 31)

- [On the (In)Effectiveness of Large Language Models for Chinese Text Correction](https://mlsanity.com/paper/2307.09007/) (2023)
- [Towards Real-World Writing Assistance: A Chinese Character Checking Benchmark with Faked and Misspelled Characters](https://mlsanity.com/paper/2311.11268/) (2023)
- [DAST: Context-Aware Compression in LLMs via Dynamic Allocation of Soft Tokens](https://mlsanity.com/paper/2502.11493/) (2025)
- [A Frustratingly Easy Plug-and-Play Detection-and-Reasoning Module for Chinese Spelling Check](https://mlsanity.com/paper/2310.09119/) (2023)
- [Disentangled Phonetic Representation for Chinese Spelling Correction](https://mlsanity.com/paper/2305.14783/) (2023)
- [Correct Like Humans: Progressive Learning Framework for Chinese Text Error Correction](https://mlsanity.com/paper/2306.17447/) (2023)
- [MESED: A Multi-modal Entity Set Expansion Dataset with Fine-grained Semantic Classes and Hard Negative Entities](https://mlsanity.com/paper/2307.14878/) (2023)
- [EcomGPT-CT: Continual Pre-training of E-commerce Large Language Models with Semi-structured Data](https://mlsanity.com/paper/2312.15696/) (2023)
- [Rethinking the Roles of Large Language Models in Chinese Grammatical Error Correction](https://mlsanity.com/paper/2402.11420/) (2024)
- [Cognitive Mismatch in Multimodal Large Language Models for Discrete Symbol Understanding](https://mlsanity.com/paper/2603.18472/) (2026)
- [CLEME: Debiasing Multi-reference Evaluation for Grammatical Error Correction](https://mlsanity.com/paper/2305.10819/) (2023)
- [Mitigating Catastrophic Forgetting in Multi-domain Chinese Spelling Correction by Multi-stage Knowledge Transfer Framework](https://mlsanity.com/paper/2402.11422/) (2024)
- [From Token to Line: Enhancing Code Generation with a Long-Term Perspective](https://mlsanity.com/paper/2504.07433/) (2025)
- [TangramPuzzle: Evaluating Multimodal Large Language Models with Compositional Spatial Reasoning](https://mlsanity.com/paper/2601.16520/) (2026)
- [Exploring the Implicit Semantic Ability of Multimodal Large Language Models: A Pilot Study on Entity Set Expansion](https://mlsanity.com/paper/2501.00330/) (2024)
- [Loss-Aware Curriculum Learning for Chinese Grammatical Error Correction](https://mlsanity.com/paper/2501.00334/) (2024)
- [MDIT: A Model-free Data Interpolation Method for Diverse Instruction Tuning](https://mlsanity.com/paper/2504.07288/) (2025)
- [Retrieval-Augmented Meta Learning for Low-Resource Text Classification](https://mlsanity.com/paper/2309.04979/) (2023)
- [MixEdit: Revisiting Data Augmentation and Beyond for Grammatical Error Correction](https://mlsanity.com/paper/2310.11671/) (2023)
- [UltraWiki: Ultra-fine-grained Entity Set Expansion with Negative Seed Entities](https://mlsanity.com/paper/2403.04247/) (2024)
- [Learning from Mistakes: Self-correct Adversarial Training for Chinese Unnatural Text Correction](https://mlsanity.com/paper/2412.17279/) (2024)
- [Chinese Spelling Correction: A Comprehensive Survey of Progress, Challenges, and Opportunities](https://mlsanity.com/paper/2502.11508/) (2025)
- [Teaching According to Talents! Instruction Tuning LLMs with Competence-Aware Curriculum Learning](https://mlsanity.com/paper/2509.13790/) (2025)
- [Prompt Learning With Knowledge Memorizing Prototypes For Generalized Few-Shot Intent Detection](https://mlsanity.com/paper/2309.04971/) (2023)
- [Refining Corpora from a Model Calibration Perspective for Chinese Spelling Correction](https://mlsanity.com/paper/2407.15498/) (2024)
- [Corrections Meet Explanations: A Unified Framework for Explainable Grammatical Error Correction](https://mlsanity.com/paper/2502.15261/) (2025)
- [A Chinese Spelling Check Framework Based on Reverse Contrastive Learning](https://mlsanity.com/paper/2210.13823/) (2022)
- [An Anchor Learning Approach for Citation Field Learning](https://mlsanity.com/paper/2309.03559/) (2023)
- [A Coin Has Two Sides: A Novel Detector-Corrector Framework for Chinese Spelling Correction](https://mlsanity.com/paper/2409.04150/) (2024)
- [EdaCSC: Two Easy Data Augmentation Methods for Chinese Spelling Correction](https://mlsanity.com/paper/2409.05105/) (2024)

## Abstract (arXiv)

Chinese Spell Checking (CSC) aims to detect and correct Chinese spelling errors. Recent researches start from the pretrained knowledge of language models and take multimodal information into CSC models to improve the performance. However, they overlook the rich knowledge in the dictionary, the reference book where one can learn how one character should be pronounced, written, and used. In this paper, we propose the LEAD framework, which renders the CSC model to learn heterogeneous knowledge from the dictionary in terms of phonetics, vision, and meaning. LEAD first constructs positive and negative samples according to the knowledge of character phonetics, glyphs, and definitions in the dictionary. Then a unified contrastive learning-based training scheme is employed to refine the representations of the CSC models. Extensive experiments and detailed analyses on the SIGHAN benchmark datasets demonstrate the effectiveness of our proposed methods.