# Glyph-aware Embedding of Chinese Characters

arXiv: https://arxiv.org/abs/1709.00028 · HTML: https://mlsanity.com/paper/1709.00028/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Aug 2017.

## Built on by (most cited first, 14 of 14)

- [Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP](https://mlsanity.com/paper/2112.10508/) (2021)
- [ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information](https://mlsanity.com/paper/2106.16038/) (2021)
- [Robust Open-Vocabulary Translation from Visual Text Representations](https://mlsanity.com/paper/2104.08211/) (2021)
- [PIXAR: Auto-Regressive Language Modeling in Pixel Space](https://mlsanity.com/paper/2401.03321/) (2024)
- [Breaking the Representation Bottleneck of Chinese Characters: Neural Machine Translation with Stroke Sequence Modeling](https://mlsanity.com/paper/2211.12781/) (2022)
- [Emergent Semantics Beyond Token Embeddings: Transformer LMs with Frozen Visual Unicode Representations](https://mlsanity.com/paper/2507.04886/) (2025)
- [Shuo Wen Jie Zi: Rethinking Dictionaries and Glyphs for Chinese Language Pre-training](https://mlsanity.com/paper/2305.18760/) (2023)
- [MIXAR: Scaling Autoregressive Pixel-based Language Models to Multiple Languages and Scripts](https://mlsanity.com/paper/2604.11575/) (2026)
- [StyleBERT: Chinese pretraining by font style information](https://mlsanity.com/paper/2202.09955/) (2022)
- [Helpful Neighbors: Leveraging Neighbors in Geographic Feature Pronunciation](https://mlsanity.com/paper/2210.10200/) (2022)
- [Logographic Information Aids Learning Better Representations for Natural Language Inference](https://mlsanity.com/paper/2211.02136/) (2022)
- [Investigating Glyph Phonetic Information for Chinese Spell Checking: What Works and What's Next](https://mlsanity.com/paper/2212.04068/) (2022)
- [On Reward Structures of Markov Decision Processes](https://mlsanity.com/paper/2308.14919/) (2023)
- [Uncertainty in Semantic Language Modeling with PIXELS](https://mlsanity.com/paper/2509.19563/) (2025)

## Abstract (arXiv)

Given the advantage and recent success of English character-level and subword-unit models in several NLP tasks, we consider the equivalent modeling problem for Chinese. Chinese script is logographic and many Chinese logograms are composed of common substructures that provide semantic, phonetic and syntactic hints. In this work, we propose to explicitly incorporate the visual appearance of a character's glyph in its representation, resulting in a novel glyph-aware embedding of Chinese characters. Being inspired by the success of convolutional neural networks in computer vision, we use them to incorporate the spatio-structural patterns of Chinese glyphs as rendered in raw pixels. In the context of two basic Chinese NLP tasks of language modeling and word segmentation, the model learns to represent each character's task-relevant semantic and syntactic information in the character-level embedding.