# Chinese Text Recognition with A Pre-Trained CLIP-Like Model Through Image-IDS Aligning

arXiv: https://arxiv.org/abs/2309.01083 · HTML: https://mlsanity.com/paper/2309.01083/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 41 references that are in the map.
- First posted on arXiv 3 Sep 2023.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Network In Network](https://mlsanity.com/paper/1312.4400/) (2013)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [Read Like Humans: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Recognition](https://mlsanity.com/paper/2103.06495/) (2021)
- [Show, Attend and Read: A Simple and Strong Baseline for Irregular Text Recognition](https://mlsanity.com/paper/1811.00751/) (2018)
- [Robust Classification with Convolutional Prototype Learning](https://mlsanity.com/paper/1805.03438/) (2018)
- [SEED: Semantics Enhanced Encoder-Decoder Framework for Scene Text Recognition](https://mlsanity.com/paper/2005.10977/) (2020)
- [MASTER: Multi-Aspect Non-local Network for Scene Text Recognition](https://mlsanity.com/paper/1910.02562/) (2019)
- [Benchmarking Chinese Text Recognition: Datasets, Baselines, and an Empirical Study](https://mlsanity.com/paper/2112.15093/) (2021)
- [Zero-Shot Chinese Character Recognition with Stroke-Level Decomposition](https://mlsanity.com/paper/2106.11613/) (2021)
- [DenseRAN for Offline Handwritten Chinese Character Recognition](https://mlsanity.com/paper/1808.04134/) (2018)
- [Multi-Column Deep Neural Networks for Offline Handwritten Chinese Character Classification](https://mlsanity.com/paper/1309.0261/) (2013)
- [2D-CTC for Scene Text Recognition](https://mlsanity.com/paper/1907.09705/) (2019)
- [Chinese Character Recognition with Radical-Structured Stroke Trees](https://mlsanity.com/paper/2211.13518/) (2022)
- [Template-Instance Loss for Offline Handwritten Chinese Character Recognition](https://mlsanity.com/paper/1910.05545/) (2019)
- [Open-set Text Recognition via Character-Context Decoupling](https://mlsanity.com/paper/2204.05535/) (2022)

## Built on by (most cited first, 26 of 26)

- [TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering](https://mlsanity.com/paper/2311.16465/) (2023)
- [SVTRv2: CTC Beats Encoder-Decoder Models in Scene Text Recognition](https://mlsanity.com/paper/2411.15858/) (2024)
- [Instruction-Guided Scene Text Recognition](https://mlsanity.com/paper/2401.17851/) (2024)
- [Out of Length Text Recognition with Sub-String Matching](https://mlsanity.com/paper/2407.12317/) (2024)
- [AnyTrans: Translate AnyText in the Image with Large Scale Models](https://mlsanity.com/paper/2406.11432/) (2024)
- [Linguistics-aware Masked Image Modeling for Self-supervised Scene Text Recognition](https://mlsanity.com/paper/2503.18746/) (2025)
- [MDiff4STR: Mask Diffusion Model for Scene Text Recognition](https://mlsanity.com/paper/2512.01422/) (2025)
- [Decoder Pre-Training with only Text for Scene Text Recognition](https://mlsanity.com/paper/2408.05706/) (2024)
- [MegaHan97K: A Large-Scale Dataset for Mega-Category Chinese Character Recognition with over 97K Categories](https://mlsanity.com/paper/2506.04807/) (2025)
- [MCCD: A Multi-Attribute Chinese Calligraphy Character Dataset Annotated with Script Styles, Dynasties, and Calligraphers](https://mlsanity.com/paper/2507.06948/) (2025)
- [Interpretable Oracle Bone Script Decipherment through Radical and Pictographic Analysis with LVLMs](https://mlsanity.com/paper/2508.10113/) (2025)
- [Restore Text First, Enhance Image Later: Two-Stage Scene Text Image Super-Resolution with Glyph Structure Guidance](https://mlsanity.com/paper/2510.21590/) (2025)
- [Improving Chinese Character Representation with Formation Tree](https://mlsanity.com/paper/2404.12693/) (2024)
- [CoLa: Chinese Character Decomposition with Compositional Latent Components](https://mlsanity.com/paper/2506.03798/) (2025)
- [BIRD: Bronze Inscription Restoration and Dating](https://mlsanity.com/paper/2511.01589/) (2025)
- [Zero-Shot Chinese Character Recognition with Hierarchical Multi-Granularity Image-Text Aligning](https://mlsanity.com/paper/2505.24837/) (2025)
- [Improving Handwritten Text Recognition via 3D Attention and Multi-Scale Training](https://mlsanity.com/paper/2410.18374/) (2024)
- [Frame-Difference Guided Dynamic Region Perception for CLIP Adaptation in Text-Video Retrieval](https://mlsanity.com/paper/2510.21806/) (2025)
- [Entropy-Aware Structural Alignment for Zero-Shot Handwritten Chinese Character Recognition](https://mlsanity.com/paper/2602.03913/) (2026)
- [Multi-Modal Character Localization and Extraction for Chinese Text Recognition](https://mlsanity.com/paper/2603.13886/) (2026)
- [Zero-Shot Chinese Character Recognition via Global-Local Dual-Branch Alignment and Hierarchical Inference](https://mlsanity.com/paper/2605.08814/) (2026)
- [Logographic Character Visual Pretraining via Semantic-based Contrastive Learning](https://mlsanity.com/paper/2608.00096/) (2026)
- [IterCAD: Iterative Program Repair for CAD Code Generation from Orthographic Views](https://mlsanity.com/paper/2608.24020/) (2026)
- [Decompose Radicals, Then Reward: Fine-Grained Inspection for Accurate Chinese Text Rendering](https://mlsanity.com/paper/2609.37569/) (2026)
- [Anchor and Adapt: Asymmetric Prompt Adaptation for Few-Shot Industrial Anomaly Detection](https://mlsanity.com/paper/2610.07016/) (2026)
- [TextVerifier: Seeing Beyond Recognition to Verify Chinese Text Rendering](https://mlsanity.com/paper/2610.19551/) (2026)

## Abstract (arXiv)

Scene text recognition has been studied for decades due to its broad applications. However, despite Chinese characters possessing different characteristics from Latin characters, such as complex inner structures and large categories, few methods have been proposed for Chinese Text Recognition (CTR). Particularly, the characteristic of large categories poses challenges in dealing with zero-shot and few-shot Chinese characters. In this paper, inspired by the way humans recognize Chinese texts, we propose a two-stage framework for CTR. Firstly, we pre-train a CLIP-like model through aligning printed character images and Ideographic Description Sequences (IDS). This pre-training stage simulates humans recognizing Chinese characters and obtains the canonical representation of each character. Subsequently, the learned representations are employed to supervise the CTR model, such that traditional single-character recognition can be improved to text-line recognition through image-IDS matching. To evaluate the effectiveness of the proposed method, we conduct extensive experiments on both Chinese character recognition (CCR) and CTR. The experimental results demonstrate that the proposed method performs best in CCR and outperforms previous methods in most scenarios of the CTR benchmark. It is worth noting that the proposed method can recognize zero-shot Chinese characters in text images without fine-tuning, whereas previous methods require fine-tuning when new classes appear. The code is available at https://github.com/FudanVI/FudanOCR/tree/main/image-ids-CTR.