Zero-Shot Chinese Character Recognition via Global-Local Dual-Branch Alignment and Hierarchical Inference
Not on the map. Why?
Abstract (arXiv)
Chinese character categories are extremely large, and unseen characters frequently arise in open-world scenarios, making zero-shot Chinese character recognition an important yet challenging problem. Existing IDS-based retrieval methods usually encode a character image and its ideographic description sequence into a single global vector for matching. Although efficient, such holistic alignment often under-models local component differences. Moreover, directly introducing patch-token level fine-grained interaction suffers from both the noise of structural operators in IDS and the high cost of full-candidate retrieval.To address these issues, we propose a Global-Local Hierarchical Perception Network (GL-HPN), which jointly learns global and local representations of character images and IDS sequences within a unified cross-modal alignment framework. The global branch supports efficient coarse recall, while the local branch improves component-level discrimination through patch-token interaction. We further introduce a structure filtering mask to suppress structurally meaningful but visually non-entity IDS operators in local similarity aggregation. On top of this, we design a coarse-to-fine hierarchical inference strategy that performs global retrieval over the full candidate set and local reranking only on Top-K candidates, followed by parameter-free multiplicative fusion of normalized posterior scores. Experimental results show that GL-HPN achieves competitive performance across multiple zero-shot splits, performs especially well under low-resource settings, and substantially reduces the inference cost of large-scale candidate retrieval.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 24 references that are in the map. First posted on arXiv 9 May 2026.
Builds on
14 of 24 references.
References · 14
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 627
- 2021Benchmarking Chinese Text Recognition: Datasets, Baselines, and an Empirical Studycited by 53
- 2023Chinese Text Recognition with A Pre-Trained CLIP-Like Model Through Image-IDS Aligningcited by 26
- 2018DenseRAN for Offline Handwritten Chinese Character Recognitioncited by 24
- 2025FineLIP: Extending CLIP's Reach via Fine-Grained Alignment with Longer Text Inputscited by 23
- 2022RZCR: Zero-shot Character Recognition via Radical-based Reasoningcited by 12
- 2022Chinese Character Recognition with Radical-Structured Stroke Treescited by 12
- 2023Toward Zero-shot Character Recognition: A Gold Standard Dataset with Radical-level Annotationscited by 10
- 2023A semantics-driven methodology for high-quality image annotationcited by 6
- 2024HierCode: A Lightweight Hierarchical Codebook for Zero-shot Chinese Text Recognitioncited by 6
- 2022STAR: Zero-Shot Chinese Character Recognition with Stroke- and Radical-Level Decompositionscited by 5
2 more
- 2022Building a visual semantics aware object hierarchycited by 3
- 2024Improving Chinese Character Representation with Formation Treecited by 2
Built on by
Not cited yet.