mlsanityData to 7 Oct 2026

Read, Listen, and See: Leveraging Multimodal Information Helps Chinese Spell Checking

arXiv 2105.12306 ↗First posted 26 May 2021cs.CLcited by 52

Not on the map. Why?

Abstract (arXiv)

Chinese Spell Checking (CSC) aims to detect and correct erroneous characters for user-generated text in the Chinese language. Most of the Chinese spelling errors are misused semantically, phonetically or graphically similar characters. Previous attempts noticed this phenomenon and try to use the similarity for this task. However, these methods use either heuristics or handcrafted confusion sets to predict the correct character. In this paper, we propose a Chinese spell checker called ReaLiSe, by directly leveraging the multimodal information of the Chinese characters. The ReaLiSe model tackles the CSC task by (1) capturing the semantic, phonetic and graphic information of the input characters, and (2) selectively mixing the information in these modalities to predict the correct output. Experiments on the SIGHAN benchmarks show that the proposed model outperforms strong baselines by a large margin.

Key facts. Cited by 52 papers in the map (data through 7 Oct 2026). Builds on 21 of its 54 references that are in the map. First posted on arXiv 26 May 2021.

Builds on

21 of 54 references.

References · 21

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2016Layer Normalizationcited by 5,579
  7. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  8. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  9. 2015VQA: Visual Question Answeringcited by 2,489
  10. 2017Decoupled Weight Decay Regularizationcited by 1,722
  11. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  12. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
9 more
  1. 2015Pointer Networkscited by 882
  2. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 668
  3. 2019Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-trainingcited by 451
  4. 2019ERNIE 2.0: A Continual Pre-training Framework for Language Understandingcited by 230
  5. 2020Spelling Error Correction with Soft-Masked BERTcited by 90
  6. 2020SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Checkcited by 57
  7. 2019Glyce: Glyph-vectors for Chinese Character Representationscited by 46
  8. 2016Multimodal Pivots for Image Caption Translationcited by 18
  9. 2017Being Negative but Constructively: Lessons Learnt from Creating Better Visual Question Answering Datasetscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 18; 2023: 17; 2024: 10; 2025: 5; 2026: 2.

52 papers.

Sort:

By area:

  1. 2022The Past Mistake is the Future Wisdom: Error-driven Contrastive Probability Optimization for Chinese Spell Checkingcited by 56
  2. 2023On the (In)Effectiveness of Large Language Models for Chinese Text Correctioncited by 53
  3. 2022Learning from the Dictionary: Heterogeneous Knowledge Guided Fine-tuning for Chinese Spell Checkingcited by 31
  4. 2022General and Domain Adaptive Chinese Spelling Check with Error Consistent Pretrainingcited by 25
  5. 2023Towards Real-World Writing Assistance: A Chinese Character Checking Benchmark with Faked and Misspelled Characterscited by 21
  6. 2022Improving Chinese Spelling Check by Character Pronunciation Prediction: The Effects of Adaptivity and Granularitycited by 19
  7. 2023Rethinking Masked Language Modeling for Chinese Spelling Correctioncited by 19
  8. 2023A Frustratingly Easy Plug-and-Play Detection-and-Reasoning Module for Chinese Spelling Checkcited by 15
  9. 2022Contextual Similarity is More Valuable than Character Similarity: An Empirical Study for Chinese Spell Checkingcited by 14
  10. 2023Disentangled Phonetic Representation for Chinese Spelling Correctioncited by 14
  11. 2023Correct Like Humans: Progressive Learning Framework for Chinese Text Error Correctioncited by 13
  12. 2024C-LLM: Learn to Check Chinese Spelling Errors Character by Charactercited by 12
38 more
  1. 2022Exploring and Adapting Chinese GPT to Pinyin Input Methodcited by 8
  2. 2022MCSCSet: A Specialist-annotated Dataset for Medical-domain Chinese Spelling Correctioncited by 8
  3. 2024Rich Semantic Knowledge Enhanced Large Language Models for Few-shot Chinese Spell Checkingcited by 7
  4. 2024A Simple yet Effective Training-free Prompt-free Approach to Chinese Spelling Correction Based on Large Language Modelscited by 7
  5. 2022Cross-Modal ASR Post-Processing System for Error Correction and Utterance Rejectioncited by 6
  6. 2022Overview of CTC 2021: Chinese Text Correction for Native Speakerscited by 6
  7. 2022Unsupervised domain adaptation for speech recognition with unsupervised error correctioncited by 5
  8. 2023An Error-Guided Correction Model for Chinese Spelling Error Correctioncited by 5
  9. 2025Mixture of Small and Large Models for Chinese Spelling Checkcited by 5
  10. 2022SDCL: Self-Distillation Contrastive Learning for Chinese Spell Checkingcited by 4
  11. 2022Error-Robust Retrieval for Chinese Spelling Checkcited by 4
  12. 2024Mitigating Catastrophic Forgetting in Multi-domain Chinese Spelling Correction by Multi-stage Knowledge Transfer Frameworkcited by 4
  13. 2025A Training-free LLM-based Approach to General Chinese Character Error Correctioncited by 4
  14. 2023A Comprehensive Evaluation and Analysis Study for Chinese Spelling Checkcited by 3
  15. 2023Does Correction Remain A Problem For Large Language Models?cited by 3
  16. 2023Enhancing conversational quality in language learning chatbots: An evaluation of GPT4 for ASR error correctioncited by 2
  17. 2023RSpell: Retrieval-augmented Framework for Domain Adaptive Chinese Spelling Checkcited by 2
  18. 2025Chinese Spelling Correction: A Comprehensive Survey of Progress, Challenges, and Opportunitiescited by 2
  19. 2022Pretraining Chinese BERT for Detecting Word Insertion and Deletion Errorscited by 1
  20. 2022AiM: Taking Answers in Mind to Correct Chinese Cloze Tests in Educational Applicationscited by 1
  21. 2023Dynamic Multi-View Fusion Mechanism For Chinese Relation Extractioncited by 1
  22. 2023Boosting Chinese ASR Error Correction with Dynamic Error Scaling Mechanismcited by 1
  23. 2024Bi-DCSpell: A Bi-directional Detector-Corrector Interactive Framework for Chinese Spelling Checkcited by 1
  24. 2024Refining Corpora from a Model Calibration Perspective for Chinese Spelling Correctioncited by 1
  25. 2024PERL: Pinyin Enhanced Rephrasing Language Model for Chinese ASR N-best Error Correctioncited by 1
  26. 2024DISC: Plug-and-Play Decoding Intervention with Similarity of Characters for Chinese Spelling Checkcited by 1
  27. 2025Unveiling the Impact of Multimodal Features on Chinese Spelling Correction: From Analysis to Designcited by 1
  28. 2022KE-QI: A Knowledge Enhanced Article Quality Identification Datasetcited by 0
  29. 2022Revising Image-Text Retrieval via Multi-Modal Entailmentcited by 0
  30. 2022A Chinese Spelling Check Framework Based on Reverse Contrastive Learningcited by 0
  31. 2022Investigating Glyph Phonetic Information for Chinese Spell Checking: What Works and What's Nextcited by 0
  32. 2023Block the Label and Noise: An N-Gram Masked Speller for Chinese Spell Checkingcited by 0
  33. 2023Exploring the Usage of Chinese Pinyin in Pretrainingcited by 0
  34. 2023Eval-GCSC: A New Metric for Evaluating ChatGPT's Performance in Chinese Spelling Correctioncited by 0
  35. 2023Make BERT-based Chinese Spelling Check Model Enhanced by Layerwise Attention and Gaussian Mixture Modelcited by 0
  36. 2024A Coin Has Two Sides: A Novel Detector-Corrector Framework for Chinese Spelling Correctioncited by 0
  37. 2024EdaCSC: Two Easy Data Augmentation Methods for Chinese Spelling Correctioncited by 0
  38. 2025RAIR: Retrieval-Augmented Iterative Refinement for Chinese Spelling Correctioncited by 0

50 most cited of 52 All 52 (JSON)