# Zero-Shot Chinese Character Recognition via Global-Local Dual-Branch Alignment and Hierarchical Inference

arXiv: https://arxiv.org/abs/2605.08814 · HTML: https://mlsanity.com/paper/2605.08814/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 24 references that are in the map.
- First posted on arXiv 9 May 2026.

## Builds on (most cited first, 14 of 14)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [FILIP: Fine-grained Interactive Language-Image Pre-Training](https://mlsanity.com/paper/2111.07783/) (2021)
- [Benchmarking Chinese Text Recognition: Datasets, Baselines, and an Empirical Study](https://mlsanity.com/paper/2112.15093/) (2021)
- [Chinese Text Recognition with A Pre-Trained CLIP-Like Model Through Image-IDS Aligning](https://mlsanity.com/paper/2309.01083/) (2023)
- [DenseRAN for Offline Handwritten Chinese Character Recognition](https://mlsanity.com/paper/1808.04134/) (2018)
- [FineLIP: Extending CLIP's Reach via Fine-Grained Alignment with Longer Text Inputs](https://mlsanity.com/paper/2504.01916/) (2025)
- [RZCR: Zero-shot Character Recognition via Radical-based Reasoning](https://mlsanity.com/paper/2207.05842/) (2022)
- [Chinese Character Recognition with Radical-Structured Stroke Trees](https://mlsanity.com/paper/2211.13518/) (2022)
- [Toward Zero-shot Character Recognition: A Gold Standard Dataset with Radical-level Annotations](https://mlsanity.com/paper/2308.00655/) (2023)
- [A semantics-driven methodology for high-quality image annotation](https://mlsanity.com/paper/2307.14119/) (2023)
- [HierCode: A Lightweight Hierarchical Codebook for Zero-shot Chinese Text Recognition](https://mlsanity.com/paper/2403.13761/) (2024)
- [STAR: Zero-Shot Chinese Character Recognition with Stroke- and Radical-Level Decompositions](https://mlsanity.com/paper/2210.08490/) (2022)
- [Building a visual semantics aware object hierarchy](https://mlsanity.com/paper/2202.13021/) (2022)
- [Improving Chinese Character Representation with Formation Tree](https://mlsanity.com/paper/2404.12693/) (2024)

## Abstract (arXiv)

Chinese character categories are extremely large, and unseen characters frequently arise in open-world scenarios, making zero-shot Chinese character recognition an important yet challenging problem. Existing IDS-based retrieval methods usually encode a character image and its ideographic description sequence into a single global vector for matching. Although efficient, such holistic alignment often under-models local component differences. Moreover, directly introducing patch-token level fine-grained interaction suffers from both the noise of structural operators in IDS and the high cost of full-candidate retrieval.To address these issues, we propose a Global-Local Hierarchical Perception Network (GL-HPN), which jointly learns global and local representations of character images and IDS sequences within a unified cross-modal alignment framework. The global branch supports efficient coarse recall, while the local branch improves component-level discrimination through patch-token interaction. We further introduce a structure filtering mask to suppress structurally meaningful but visually non-entity IDS operators in local similarity aggregation. On top of this, we design a coarse-to-fine hierarchical inference strategy that performs global retrieval over the full candidate set and local reranking only on Top-K candidates, followed by parameter-free multiplicative fusion of normalized posterior scores. Experimental results show that GL-HPN achieves competitive performance across multiple zero-shot splits, performs especially well under low-resource settings, and substantially reduces the inference cost of large-scale candidate retrieval.