# Glyce: Glyph-vectors for Chinese Character Representations

arXiv: https://arxiv.org/abs/1901.10125 · HTML: https://mlsanity.com/paper/1901.10125/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jan 2019.

## Built on by (most cited first, 30 of 46)

- [ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation](https://mlsanity.com/paper/2107.02137/) (2021)
- [Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP](https://mlsanity.com/paper/2112.10508/) (2021)
- [Expressive Text-to-Image Generation with Rich Text](https://mlsanity.com/paper/2304.06720/) (2023)
- [ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information](https://mlsanity.com/paper/2106.16038/) (2021)
- [Locate and Label: A Two-stage Identifier for Nested Named Entity Recognition](https://mlsanity.com/paper/2105.06804/) (2021)
- [Language Modelling with Pixels](https://mlsanity.com/paper/2207.06991/) (2022)
- [Read, Listen, and See: Leveraging Multimodal Information Helps Chinese Spell Checking](https://mlsanity.com/paper/2105.12306/) (2021)
- [Robust Open-Vocabulary Translation from Visual Text Representations](https://mlsanity.com/paper/2104.08211/) (2021)
- [Lexicon Enhanced Chinese Sequence Labeling Using BERT Adapter](https://mlsanity.com/paper/2105.07148/) (2021)
- [TaCL: Improving BERT Pre-training with Token-aware Contrastive Learning](https://mlsanity.com/paper/2111.04198/) (2021)
- [General and Domain Adaptive Chinese Spelling Check with Error Consistent Pretraining](https://mlsanity.com/paper/2203.10929/) (2022)
- [Parallel Instance Query Network for Named Entity Recognition](https://mlsanity.com/paper/2203.10545/) (2022)
- [PIXAR: Auto-Regressive Language Modeling in Pixel Space](https://mlsanity.com/paper/2401.03321/) (2024)
- [Universal Multimodal Representation for Language Understanding](https://mlsanity.com/paper/2301.03344/) (2023)
- [Improving Language Understanding from Screenshots](https://mlsanity.com/paper/2402.14073/) (2024)
- [MECT: Multi-Metadata Embedding based Cross-Transformer for Chinese Named Entity Recognition](https://mlsanity.com/paper/2107.05418/) (2021)
- [Sub-Character Tokenization for Chinese Pretrained Language Models](https://mlsanity.com/paper/2106.00400/) (2021)
- [Masked Conditional Random Fields for Sequence Labeling](https://mlsanity.com/paper/2103.10682/) (2021)
- [GlyphCRM: Bidirectional Encoder Representation for Chinese Character with its Glyph](https://mlsanity.com/paper/2107.00395/) (2021)
- [DyLex: Incorporating Dynamic Lexicons into BERT for Sequence Labeling](https://mlsanity.com/paper/2109.08818/) (2021)
- [Enhanced Language Representation with Label Knowledge for Span Extraction](https://mlsanity.com/paper/2111.00884/) (2021)
- [Robust Self-Augmentation for Named Entity Recognition with Meta Reweighting](https://mlsanity.com/paper/2204.11406/) (2022)
- [GNN-SL: Sequence Labeling Based on Nearest Examples via GNN](https://mlsanity.com/paper/2212.02017/) (2022)
- [TransCouplet:Transformer based Chinese Couplet Generation](https://mlsanity.com/paper/2112.01707/) (2021)
- [Unsupervised Boundary-Aware Language Model Pretraining for Chinese Sequence Labeling](https://mlsanity.com/paper/2210.15231/) (2022)
- [Span-Oriented Information Extraction -- A Unifying Perspective on Information Extraction](https://mlsanity.com/paper/2403.15453/) (2024)
- [Neural Sequence Segmentation as Determining the Leftmost Segments](https://mlsanity.com/paper/2104.07217/) (2021)
- [Context Enhanced Short Text Matching using Clickthrough Data](https://mlsanity.com/paper/2203.01849/) (2022)
- [PaCaNet: A Study on CycleGAN with Transfer Learning for Diversifying Fused Chinese Painting and Calligraphy](https://mlsanity.com/paper/2301.13082/) (2023)
- [Dynamic Multi-View Fusion Mechanism For Chinese Relation Extraction](https://mlsanity.com/paper/2303.05082/) (2023)

## Abstract (arXiv)

It is intuitive that NLP tasks for logographic languages like Chinese should benefit from the use of the glyph information in those languages. However, due to the lack of rich pictographic evidence in glyphs and the weak generalization ability of standard computer vision models on character data, an effective way to utilize the glyph information remains to be found. In this paper, we address this gap by presenting Glyce, the glyph-vectors for Chinese character representations. We make three major innovations: (1) We use historical Chinese scripts (e.g., bronzeware script, seal script, traditional Chinese, etc) to enrich the pictographic evidence in characters; (2) We design CNN structures (called tianzege-CNN) tailored to Chinese character image processing; and (3) We use image-classification as an auxiliary task in a multi-task learning setup to increase the model's ability to generalize. We show that glyph-based models are able to consistently outperform word/char ID-based models in a wide range of Chinese NLP tasks. We are able to set new state-of-the-art results for a variety of Chinese NLP tasks, including tagging (NER, CWS, POS), sentence pair classification, single sentence classification tasks, dependency parsing, and semantic role labeling. For example, the proposed model achieves an F1 score of 80.6 on the OntoNotes dataset of NER, +1.5 over BERT; it achieves an almost perfect accuracy of 99.8% on the Fudan corpus for text classification. Code found at https://github.com/ShannonAI/glyce.