mlsanityData to 14 Oct 2026

Symmetrical Linguistic Feature Distillation with CLIP for Scene Text Recognition

arXiv 2310.04999 ↗First posted 8 Oct 2023cs.CVcited by 14

On the map: Knowledge distillation → Multimodal distillation

Abstract (arXiv)

In this paper, we explore the potential of the Contrastive Language-Image Pretraining (CLIP) model in scene text recognition (STR), and establish a novel Symmetrical Linguistic Feature Distillation framework (named CLIP-OCR) to leverage both visual and linguistic knowledge in CLIP. Different from previous CLIP-based methods mainly considering feature generalization on visual encoding, we propose a symmetrical distillation strategy (SDS) that further captures the linguistic knowledge in the CLIP text encoder. By cascading the CLIP image encoder with the reversed CLIP text encoder, a symmetrical structure is built with an image-to-text feature flow that covers not only visual but also linguistic information for distillation.Benefiting from the natural alignment in CLIP, such guidance flow provides a progressive optimization objective from vision to language, which can supervise the STR feature forwarding process layer-by-layer.Besides, a new Linguistic Consistency Loss (LCL) is proposed to enhance the linguistic capability by considering second-order statistics during the optimization. Overall, CLIP-OCR is the first to design a smooth transition between image and text for the STR task.Extensive experiments demonstrate the effectiveness of CLIP-OCR with 93.8% average accuracy on six popular STR benchmarks.Code will be available at https://github.com/wzx99/CLIPOCR.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Multimodal distillation, one of 12 directions in Knowledge distillation. Builds on 44 of its 64 references that are in the map. First posted on arXiv 8 Oct 2023.

Builds on

44 of 64 references.

References · 44

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  7. 2021Learning to Prompt for Vision-Language Modelscited by 2,281
  8. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 616
  9. 2015An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognitioncited by 520
  10. 2021Image Segmentation Using Text and Image Promptscited by 512
  11. 2016Synthetic Data for Text Localisation in Natural Imagescited by 396
  12. 2019A Comprehensive Overhaul of Feature Distillationcited by 348
32 more
  1. 2020Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlookscited by 316
  2. 2020Single Image Super-Resolution via a Holistic Attention Networkcited by 265
  3. 2016COCO-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Imagescited by 236
  4. 2014Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognitioncited by 214
  5. 2021Read Like Humans: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Recognitioncited by 194
  6. 2019What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysiscited by 191
  7. 2022Scene Text Recognition with Permuted Autoregressive Sequence Modelscited by 149
  8. 2019ICDAR2019 Robust Reading Challenge on Arbitrary-Shaped Text (RRC-ArT)cited by 147
  9. 2022Bridging the Gap between Object and Image-level Representations for Open-Vocabulary Detectioncited by 146
  10. 2022Linearly Mapping from Image to Text Spacecited by 142
  11. 2020SEED: Semantics Enhanced Encoder-Decoder Framework for Scene Text Recognitioncited by 119
  12. 2022CLIP Models are Few-shot Learners: Empirical Studies on VQA and Visual Entailmentcited by 113
  13. 2019Decoupled Attention Network for Text Recognitioncited by 102
  14. 2021From Two to One: A New Scene Text Recognizer with Visual Language Modeling Networkcited by 97
  15. 2022Contrastive Learning Rivals Masked Image Modeling in Fine-tuning via Feature Distillationcited by 93
  16. 2022SVTR: Scene Text Recognition with a Single Visual Modelcited by 88
  17. 2020RobustScanner: Dynamically Enhancing Positional Clues for Robust Text Recognitioncited by 85
  18. 2021Vision Transformer for Fast and Efficient Scene Text Recognitioncited by 81
  19. 2019TextScanner: Reading Characters in Order for Robust Scene Text Recognitioncited by 68
  20. 2021What If We Only Use Real Datasets for Scene Text Recognition? Toward Scene Text Recognition With Fewer Labelscited by 63
  21. 2018Scene Text Recognition from Two-Dimensional Perspectivecited by 57
  22. 2022Toward Understanding WordArt: Corner-Guided Transformer for Scene Text Recognitioncited by 45
  23. 2023Turning a CLIP Model into a Scene Text Detectorcited by 44
  24. 2022ABINet++: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Spottingcited by 40
  25. 2022Multi-Granularity Prediction for Scene Text Recognitioncited by 36
  26. 2022Understanding and Improving the Role of Projection Head in Self-Supervised Learningcited by 36
  27. 2022MaskOCR: Text Recognition with Masked Encoder-Decoder Pretrainingcited by 34
  28. 2022CLIP-TD: CLIP Targeted Distillation for Vision-Language Taskscited by 32
  29. 2022Stare at What You See: Masked Image Modeling without Reconstructioncited by 30
  30. 2023Linguistic More: Taking a Further Step toward Efficient and Accurate Scene Text Recognitioncited by 18
  31. 2023CLIPTER: Looking at the Bigger Picture in Scene Text Recognitioncited by 16
  32. 20192D-CTC for Scene Text Recognitioncited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 7; 2025: 5; 2026: 2.

14 papers.

Sort:

By area:

  1. 2024Hi-SAM: Marrying Segment Anything Model for Hierarchical Text Segmentationcited by 30
  2. 2024How Control Information Influences Multilingual Text Image Generation and Editing?cited by 9
  3. 2024Focus, Distinguish, and Prompt: Unleashing CLIP for Efficient and Flexible Scene Text Retrievalcited by 6
  4. 2025Linguistics-aware Masked Image Modeling for Self-supervised Scene Text Recognitioncited by 6
  5. 2025Inversion-DPO: Precise and Efficient Post-Training for Diffusion Modelscited by 6
  6. 2024Scene-Text Grounding for Text-Based Video Question Answeringcited by 5
  7. 2024Boosting Semi-Supervised Scene Text Recognition via Viewing and Summarizingcited by 2
  8. 2024Focus on the Whole Character: Discriminative Character Modeling for Scene Text Recognitioncited by 1
  9. 2025LadderMoE: Ladder-Side Mixture of Experts Adapters for Bronze Inscription Recognitioncited by 1
  10. 2024Attacks on multimodal modelscited by 0
  11. 2025Accurate Scene Text Recognition with Efficient Model Scaling and Cloze Self-Distillationcited by 0
  12. 2025A Lightweight Multi-Module Fusion Approach for Korean Character Recognitioncited by 0
2 more
  1. 2026DocRevive: A Unified Pipeline for Document Text Restorationcited by 0
  2. 2026Masked Next-Scale Prediction for Self-supervised Scene Text Recognitioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023CLIP4STR: A Simple Baseline for Scene Text Recognition with Pre-trained Vision-Language Model
  2. 2024Decoder Pre-Training with only Text for Scene Text Recognition
  3. 2022Language Matters: A Weakly Supervised Vision-Language Pre-training Approach for Scene Text Detection and Spotting
  4. 2024A Progressive Framework of Vision-language Knowledge Distillation and Alignment for Multilingual Scene
  5. 2021Text is Text, No Matter What: Unifying Text Recognition using Knowledge Distillation
  6. 2026MLLMCLIP: Feature-Level Distillation of MLLM for Robust Vision-Language Representations
  7. 2023SILC: Improving Vision Language Pretraining with Self-Distillation
  8. 2026MEVL-STP: Multi-Encoder and Vision Language Model for Arbitrarily Shaped Scene Text Spotting