# Exploring and Adapting Chinese GPT to Pinyin Input Method

arXiv: https://arxiv.org/abs/2203.00249 · HTML: https://mlsanity.com/paper/2203.00249/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 25 references that are in the map.
- First posted on arXiv 1 Mar 2022.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information](https://mlsanity.com/paper/2106.16038/) (2021)
- [CPM: A Large-scale Generative Chinese Pre-trained Language Model](https://mlsanity.com/paper/2012.00413/) (2020)
- [Read, Listen, and See: Leveraging Multimodal Information Helps Chinese Spell Checking](https://mlsanity.com/paper/2105.12306/) (2021)
- [CLUECorpus2020: A Large-scale Chinese Corpus for Pre-training Language Model](https://mlsanity.com/paper/2003.01355/) (2020)
- [TexSmart: A Text Understanding System for Fine-Grained NER and Enhanced Semantic Analysis](https://mlsanity.com/paper/2012.15639/) (2020)
- [Tracing a Loose Wordhood for Chinese Input Method Engine](https://mlsanity.com/paper/1712.04158/) (2017)
- [Chinese Pinyin Aided IME, Input What You Have Not Keystroked Yet](https://mlsanity.com/paper/1809.00329/) (2018)

## Built on by (most cited first, 8 of 8)

- [CSCD-NS: a Chinese Spelling Check Dataset for Native Speakers](https://mlsanity.com/paper/2211.08788/) (2022)
- [Personalized Language Model Learning on Text Data Without User Identifiers](https://mlsanity.com/paper/2501.06062/) (2025)
- [Generative Input: Towards Next-Generation Input Methods Paradigm](https://mlsanity.com/paper/2311.01166/) (2023)
- [An Energy-based Model for Word-level AutoCompletion in Computer-aided Translation](https://mlsanity.com/paper/2407.20083/) (2024)
- [Dynamic Adaptive Rank Space Exploration for Efficient Sentiment Analysis with Large Language Models](https://mlsanity.com/paper/2410.16589/) (2024)
- [CNMBERT: A Model for Converting Hanyu Pinyin Abbreviations to Chinese Characters](https://mlsanity.com/paper/2411.11770/) (2024)
- [Explainable Neural Networks with Guarantees: A Sparse Estimation Approach](https://mlsanity.com/paper/2501.02010/) (2025)
- [CNM-BERT: A Drop-In Structural Embedding for Chinese Characters via Ideographic Description Sequences](https://mlsanity.com/paper/2608.05167/) (2026)

## Abstract (arXiv)

While GPT has become the de-facto method for text generation tasks, its application to pinyin input method remains unexplored. In this work, we make the first exploration to leverage Chinese GPT for pinyin input method. We find that a frozen GPT achieves state-of-the-art performance on perfect pinyin. However, the performance drops dramatically when the input includes abbreviated pinyin. A reason is that an abbreviated pinyin can be mapped to many perfect pinyin, which links to even larger number of Chinese characters. We mitigate this issue with two strategies, including enriching the context with pinyin and optimizing the training process to help distinguish homophones. To further facilitate the evaluation of pinyin input method, we create a dataset consisting of 270K instances from 15 domains. Results show that our approach improves performance on abbreviated pinyin across all domains. Model analysis demonstrates that both strategies contribute to the performance boost.