# Logographic Character Visual Pretraining via Semantic-based Contrastive Learning

arXiv: https://arxiv.org/abs/2608.00096 · HTML: https://mlsanity.com/paper/2608.00096/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 54 references that are in the map.
- First posted on arXiv 30 Jul 2026.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [VL-BERT: Pre-training of Generic Visual-Linguistic Representations](https://mlsanity.com/paper/1908.08530/) (2019)
- [Balanced Contrastive Learning for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2207.09052/) (2022)
- [Handwritten Optical Character Recognition (OCR): A Comprehensive Systematic Literature Review (SLR)](https://mlsanity.com/paper/2001.00139/) (2020)
- [Probabilistic Contrastive Learning for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2403.06726/) (2024)
- [Zero-Shot Chinese Character Recognition with Stroke-Level Decomposition](https://mlsanity.com/paper/2106.11613/) (2021)
- [Chinese Text Recognition with A Pre-Trained CLIP-Like Model Through Image-IDS Aligning](https://mlsanity.com/paper/2309.01083/) (2023)
- [Out-of-Distribution Detection in Long-Tailed Recognition with Calibrated Outlier Class Learning](https://mlsanity.com/paper/2312.10686/) (2023)
- [Online and Offline Handwritten Chinese Character Recognition: A Comprehensive Study and New Benchmark](https://mlsanity.com/paper/1606.05763/) (2016)
- [Multi-Column Deep Neural Networks for Offline Handwritten Chinese Character Classification](https://mlsanity.com/paper/1309.0261/) (2013)
- [NCL++: Nested Collaborative Learning for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2306.16709/) (2023)
- [DropSample: A New Training Method to Enhance Deep Convolutional Neural Networks for Large-Scale Unconstrained Handwritten Chinese Character Recognition](https://mlsanity.com/paper/1505.05354/) (2015)
- [RZCR: Zero-shot Character Recognition via Radical-based Reasoning](https://mlsanity.com/paper/2207.05842/) (2022)
- [RCRN: Real-world Character Image Restoration Network via Skeleton Extraction](https://mlsanity.com/paper/2207.07795/) (2022)
- [High Performance Offline Handwritten Chinese Character Recognition Using GoogLeNet and Directional Feature Maps](https://mlsanity.com/paper/1505.04925/) (2015)
- [Toward Zero-shot Character Recognition: A Gold Standard Dataset with Radical-level Annotations](https://mlsanity.com/paper/2308.00655/) (2023)
- [Hierarchical Character Embeddings: Learning Phonological and Semantic Representations in Languages of Logographic Origin using Recursive Neural Networks](https://mlsanity.com/paper/1912.09913/) (2019)

## Abstract (arXiv)

Current deep learning-based character vision studies, e.g., text recognition, character image denoising, and historical text completion, are offering new solutions for learning, managing, and utilizing character resources. However, the performance of these studies peaks only with large and balanced datasets, which is a rarity with real-world character datasets, especially for logographic character languages, e.g., Chinese. The imbalance in data distribution of logographic characters is a common issue due to differences in character usage frequency and new characters being continuously created. In this paper, we propose a novel method for logographic character recognition, which introduces a multi-modal learning approach using visual semantics and contextual semantics of characters. A novel pre-training strategy is designed to enhance deep visual representations, especially for datasets suffering from issues of imbalanced and rare instances, by extracting the contextual semantics of each character from the corresponding language models. We conduct experiments across various datasets to evaluate our character recognition method and further validate the contrastive pre-training strategy by several downstream tasks. Experimental results demonstrate the superiority of our method compared to state-of-the-art methods.