# Breaking the Representation Bottleneck of Chinese Characters: Neural Machine Translation with Stroke Sequence Modeling

arXiv: https://arxiv.org/abs/2211.12781 · HTML: https://mlsanity.com/paper/2211.12781/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 48 references that are in the map.
- First posted on arXiv 23 Nov 2022.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates](https://mlsanity.com/paper/1804.10959/) (2018)
- [Using the Output Embedding to Improve Language Models](https://mlsanity.com/paper/1608.05859/) (2016)
- [Scaling Neural Machine Translation](https://mlsanity.com/paper/1806.00187/) (2018)
- [BPE-Dropout: Simple and Effective Subword Regularization](https://mlsanity.com/paper/1910.13267/) (2019)
- [Charformer: Fast Character Transformers via Gradient-based Subword Tokenization](https://mlsanity.com/paper/2106.12672/) (2021)
- [Robust Neural Machine Translation with Doubly Adversarial Inputs](https://mlsanity.com/paper/1906.02443/) (2019)
- [AdvAug: Robust Adversarial Augmentation for Neural Machine Translation](https://mlsanity.com/paper/2006.11834/) (2020)
- [ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information](https://mlsanity.com/paper/2106.16038/) (2021)
- [Norm-Based Curriculum Learning for Neural Machine Translation](https://mlsanity.com/paper/2006.02014/) (2020)
- [Multi-view Subword Regularization](https://mlsanity.com/paper/2103.08490/) (2021)
- [How BPE Affects Memorization in Transformers](https://mlsanity.com/paper/2110.02782/) (2021)
- [Understanding and Improving Encoder Layer Fusion in Sequence-to-Sequence Learning](https://mlsanity.com/paper/2012.14768/) (2020)
- [Subword Pooling Makes a Difference](https://mlsanity.com/paper/2102.10864/) (2021)
- [On the Copying Behaviors of Pre-Training for Neural Machine Translation](https://mlsanity.com/paper/2107.08212/) (2021)
- [Glyph-aware Embedding of Chinese Characters](https://mlsanity.com/paper/1709.00028/) (2017)
- [On the Complementarity between Pre-Training and Back-Translation for Neural Machine Translation](https://mlsanity.com/paper/2110.01811/) (2021)
- [Chinese Character Decomposition for Neural MT with Multi-Word Expressions](https://mlsanity.com/paper/2104.04497/) (2021)
- [State-of-the-art Chinese Word Segmentation with Bi-LSTMs](https://mlsanity.com/paper/1808.06511/) (2018)
- [Shared-Private Bilingual Word Embeddings for Neural Machine Translation](https://mlsanity.com/paper/1906.03100/) (2019)
- [CipherDAug: Ciphertext based Data Augmentation for Neural Machine Translation](https://mlsanity.com/paper/2204.00665/) (2022)

## Built on by (most cited first, 6 of 6)

- [A Human-Computer Collaborative Tool for Training a Single Large Language Model Agent into a Network through Few Examples](https://mlsanity.com/paper/2404.15974/) (2024)
- [ConsistTL: Modeling Consistency in Transfer Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2212.04262/) (2022)
- [Clustering Pseudo Language Family in Multilingual Translation Models with Fisher Information Matrix](https://mlsanity.com/paper/2312.02820/) (2023)
- [Multi-Modal Multi-Granularity Tokenizer for Chu Bamboo Slip Scripts](https://mlsanity.com/paper/2409.01011/) (2024)
- [MultiLexNorm++: A Unified Benchmark and a Generative Model for Lexical Normalization for Asian Languages](https://mlsanity.com/paper/2601.16623/) (2026)
- [Enabling Stroke-Level Structural Analysis of Hieroglyphic Scripts without Language-Specific Priors](https://mlsanity.com/paper/2601.05508/) (2026)

## Abstract (arXiv)

Existing research generally treats Chinese character as a minimum unit for representation. However, such Chinese character representation will suffer two bottlenecks: 1) Learning bottleneck, the learning cannot benefit from its rich internal features (e.g., radicals and strokes); and 2) Parameter bottleneck, each individual character has to be represented by a unique vector. In this paper, we introduce a novel representation method for Chinese characters to break the bottlenecks, namely StrokeNet, which represents a Chinese character by a Latinized stroke sequence (e.g., "ao1 (concave)" to "ajaie" and "tu1 (convex)" to "aeaqe"). Specifically, StrokeNet maps each stroke to a specific Latin character, thus allowing similar Chinese characters to have similar Latin representations. With the introduction of StrokeNet to neural machine translation (NMT), many powerful but not applicable techniques to non-Latin languages (e.g., shared subword vocabulary learning and ciphertext-based data augmentation) can now be perfectly implemented. Experiments on the widely-used NIST Chinese-English, WMT17 Chinese-English and IWSLT17 Japanese-English NMT tasks show that StrokeNet can provide a significant performance boost over the strong baselines with fewer model parameters, achieving 26.5 BLEU on the WMT17 Chinese-English task which is better than any previously reported results without using monolingual data. Code and scripts are freely available at https://github.com/zjwang21/StrokeNet.