# Symmetrical Linguistic Feature Distillation with CLIP for Scene Text Recognition arXiv: https://arxiv.org/abs/2310.04999 ยท HTML: https://mlsanity.com/paper/2310.04999/ ## Key facts - Cited by 14 papers in the map (data through 14 Oct 2026). - Placed in Multimodal distillation, one of 12 directions in Knowledge distillation. - Builds on 44 of its 64 references that are in the map. - First posted on arXiv 8 Oct 2023. ## Placed in - Knowledge distillation / [Multimodal distillation](https://mlsanity.com/distill/multimodal-distillation/) ## Builds on (most cited first, 30 of 44) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014) - [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015) - [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018) - [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022) - [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022) - [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021) - [RegionCLIP: Region-based Language-Image Pretraining](https://mlsanity.com/paper/2112.09106/) (2021) - [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015) - [Image Segmentation Using Text and Image Prompts](https://mlsanity.com/paper/2112.10003/) (2021) - [Synthetic Data for Text Localisation in Natural Images](https://mlsanity.com/paper/1604.06646/) (2016) - [A Comprehensive Overhaul of Feature Distillation](https://mlsanity.com/paper/1904.01866/) (2019) - [Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks](https://mlsanity.com/paper/2004.05937/) (2020) - [Single Image Super-Resolution via a Holistic Attention Network](https://mlsanity.com/paper/2008.08767/) (2020) - [COCO-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Images](https://mlsanity.com/paper/1601.07140/) (2016) - [Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition](https://mlsanity.com/paper/1406.2227/) (2014) - [Read Like Humans: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Recognition](https://mlsanity.com/paper/2103.06495/) (2021) - [What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysis](https://mlsanity.com/paper/1904.01906/) (2019) - [Scene Text Recognition with Permuted Autoregressive Sequence Models](https://mlsanity.com/paper/2207.06966/) (2022) - [ICDAR2019 Robust Reading Challenge on Arbitrary-Shaped Text (RRC-ArT)](https://mlsanity.com/paper/1909.07145/) (2019) - [Bridging the Gap between Object and Image-level Representations for Open-Vocabulary Detection](https://mlsanity.com/paper/2207.03482/) (2022) - [Linearly Mapping from Image to Text Space](https://mlsanity.com/paper/2209.15162/) (2022) - [SEED: Semantics Enhanced Encoder-Decoder Framework for Scene Text Recognition](https://mlsanity.com/paper/2005.10977/) (2020) - [CLIP Models are Few-shot Learners: Empirical Studies on VQA and Visual Entailment](https://mlsanity.com/paper/2203.07190/) (2022) - [Decoupled Attention Network for Text Recognition](https://mlsanity.com/paper/1912.10205/) (2019) - [From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network](https://mlsanity.com/paper/2108.09661/) (2021) - [Contrastive Learning Rivals Masked Image Modeling in Fine-tuning via Feature Distillation](https://mlsanity.com/paper/2205.14141/) (2022) - [SVTR: Scene Text Recognition with a Single Visual Model](https://mlsanity.com/paper/2205.00159/) (2022) - [RobustScanner: Dynamically Enhancing Positional Clues for Robust Text Recognition](https://mlsanity.com/paper/2007.07542/) (2020) - [Vision Transformer for Fast and Efficient Scene Text Recognition](https://mlsanity.com/paper/2105.08582/) (2021) ## Built on by (most cited first, 14 of 14) - [Hi-SAM: Marrying Segment Anything Model for Hierarchical Text Segmentation](https://mlsanity.com/paper/2401.17904/) (2024) - [How Control Information Influences Multilingual Text Image Generation and Editing?](https://mlsanity.com/paper/2407.11502/) (2024) - [Focus, Distinguish, and Prompt: Unleashing CLIP for Efficient and Flexible Scene Text Retrieval](https://mlsanity.com/paper/2408.00441/) (2024) - [Linguistics-aware Masked Image Modeling for Self-supervised Scene Text Recognition](https://mlsanity.com/paper/2503.18746/) (2025) - [Inversion-DPO: Precise and Efficient Post-Training for Diffusion Models](https://mlsanity.com/paper/2507.11554/) (2025) - [Scene-Text Grounding for Text-Based Video Question Answering](https://mlsanity.com/paper/2409.14319/) (2024) - [Boosting Semi-Supervised Scene Text Recognition via Viewing and Summarizing](https://mlsanity.com/paper/2411.15585/) (2024) - [Focus on the Whole Character: Discriminative Character Modeling for Scene Text Recognition](https://mlsanity.com/paper/2407.05562/) (2024) - [LadderMoE: Ladder-Side Mixture of Experts Adapters for Bronze Inscription Recognition](https://mlsanity.com/paper/2510.01651/) (2025) - [Attacks on multimodal models](https://mlsanity.com/paper/2412.01725/) (2024) - [Accurate Scene Text Recognition with Efficient Model Scaling and Cloze Self-Distillation](https://mlsanity.com/paper/2503.16184/) (2025) - [A Lightweight Multi-Module Fusion Approach for Korean Character Recognition](https://mlsanity.com/paper/2504.05770/) (2025) - [DocRevive: A Unified Pipeline for Document Text Restoration](https://mlsanity.com/paper/2604.10077/) (2026) - [Masked Next-Scale Prediction for Self-supervised Scene Text Recognition](https://mlsanity.com/paper/2605.14885/) (2026) ## Abstract (arXiv) In this paper, we explore the potential of the Contrastive Language-Image Pretraining (CLIP) model in scene text recognition (STR), and establish a novel Symmetrical Linguistic Feature Distillation framework (named CLIP-OCR) to leverage both visual and linguistic knowledge in CLIP. Different from previous CLIP-based methods mainly considering feature generalization on visual encoding, we propose a symmetrical distillation strategy (SDS) that further captures the linguistic knowledge in the CLIP text encoder. By cascading the CLIP image encoder with the reversed CLIP text encoder, a symmetrical structure is built with an image-to-text feature flow that covers not only visual but also linguistic information for distillation.Benefiting from the natural alignment in CLIP, such guidance flow provides a progressive optimization objective from vision to language, which can supervise the STR feature forwarding process layer-by-layer.Besides, a new Linguistic Consistency Loss (LCL) is proposed to enhance the linguistic capability by considering second-order statistics during the optimization. Overall, CLIP-OCR is the first to design a smooth transition between image and text for the STR task.Extensive experiments demonstrate the effectiveness of CLIP-OCR with 93.8% average accuracy on six popular STR benchmarks.Code will be available at https://github.com/wzx99/CLIPOCR.