# Hierarchical Character Embeddings: Learning Phonological and Semantic Representations in Languages of Logographic Origin using Recursive Neural Networks

arXiv: https://arxiv.org/abs/1912.09913 · HTML: https://mlsanity.com/paper/1912.09913/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Dec 2019.

## Built on by (most cited first, 8 of 8)

- [Tree-constrained Pointer Generator with Graph Neural Network Encodings for Contextual Speech Recognition](https://mlsanity.com/paper/2207.00857/) (2022)
- [Breaking the Representation Bottleneck of Chinese Characters: Neural Machine Translation with Stroke Sequence Modeling](https://mlsanity.com/paper/2211.12781/) (2022)
- [Graph Neural Networks for Contextual ASR with the Tree-Constrained Pointer Generator](https://mlsanity.com/paper/2305.18824/) (2023)
- [Generic Multi-modal Representation Learning for Network Traffic Analysis](https://mlsanity.com/paper/2405.02649/) (2024)
- [Helpful Neighbors: Leveraging Neighbors in Geographic Feature Pronunciation](https://mlsanity.com/paper/2210.10200/) (2022)
- [Finite-context Indexing of Restricted Output Space for NLP Models Facing Noisy Input](https://mlsanity.com/paper/2310.14110/) (2023)
- [Enabling Stroke-Level Structural Analysis of Hieroglyphic Scripts without Language-Specific Priors](https://mlsanity.com/paper/2601.05508/) (2026)
- [Logographic Character Visual Pretraining via Semantic-based Contrastive Learning](https://mlsanity.com/paper/2608.00096/) (2026)

## Abstract (arXiv)

Logographs (Chinese characters) have recursive structures (i.e. hierarchies of sub-units in logographs) that contain phonological and semantic information, as developmental psychology literature suggests that native speakers leverage on the structures to learn how to read. Exploiting these structures could potentially lead to better embeddings that can benefit many downstream tasks. We propose building hierarchical logograph (character) embeddings from logograph recursive structures using treeLSTM, a recursive neural network. Using recursive neural network imposes a prior on the mapping from logographs to embeddings since the network must read in the sub-units in logographs according to the order specified by the recursive structures. Based on human behavior in language learning and reading, we hypothesize that modeling logographs' structures using recursive neural network should be beneficial. To verify this claim, we consider two tasks (1) predicting logographs' Cantonese pronunciation from logographic structures and (2) language modeling. Empirical results show that the proposed hierarchical embeddings outperform baseline approaches. Diagnostic analysis suggests that hierarchical embeddings constructed using treeLSTM is less sensitive to distractors, thus is more robust, especially on complex logographs.