mlsanityData to 14 Oct 2026

CNMBERT: A Model for Converting Hanyu Pinyin Abbreviations to Chinese Characters

arXiv 2411.11770 ↗First posted 18 Nov 2024cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

The task of converting Hanyu Pinyin abbreviations to Chinese characters is a significant branch within the domain of Chinese Spelling Correction (CSC). It plays an important role in many downstream applications such as named entity recognition and sentiment analysis. This task typically involves text-length alignment and seems easy to solve; however, due to the limited information content in pinyin abbreviations, achieving accurate conversion is challenging. In this paper, we treat this as a fill-mask task and propose CNMBERT, which stands for zh-CN Pinyin Multi-mask BERT Model, as a solution to this issue. By introducing a multi-mask strategy and Mixture of Experts (MoE) layers, CNMBERT outperforms fine-tuned GPT models and ChatGPT-4o with a 61.53% MRR score and 51.86% accuracy on a 10,373-sample test dataset.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 25 references that are in the map. First posted on arXiv 18 Nov 2024.

Builds on

17 of 25 references.

References · 17

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  5. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,475
  6. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  7. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  8. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 867
  9. 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scalecited by 404
  10. 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 305
  11. 2020Revisiting Pre-Trained Models for Chinese Natural Language Processingcited by 266
  12. 2018Deep Learning for Sentiment Analysis : A Surveycited by 139
5 more
  1. 2021Unified Named Entity Recognition as Word-Word Relation Classificationcited by 90
  2. 2020N-LTP: An Open-source Neural Language Technology Platform for Chinesecited by 29
  3. 2019PKUSEG: A Toolkit for Multi-Domain Chinese Word Segmentationcited by 17
  4. 2023Chinese Spelling Correction as Rephrasing Language Modelcited by 15
  5. 2022Exploring and Adapting Chinese GPT to Pinyin Input Methodcited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Disentangled Phonetic Representation for Chinese Spelling Correction
  2. 2023An Error-Guided Correction Model for Chinese Spelling Error Correction
  3. 2025Chinese Spelling Correction: A Comprehensive Survey of Progress, Challenges, and Opportunities
  4. 2023Rethinking Masked Language Modeling for Chinese Spelling Correction
  5. 2025RAIR: Retrieval-Augmented Iterative Refinement for Chinese Spelling Correction
  6. 2019Enhancing Pre-trained Chinese Character Representation with Word-aligned Attention
  7. 2025LLM Encoder vs. Decoder: Robust Detection of Chinese AI-Generated Text with LoRA
  8. 2023RSpell: Retrieval-augmented Framework for Domain Adaptive Chinese Spelling Check