mlsanityData to 13 Oct 2026

SAViL-Det: Semantic-Aware Vision-Language Model for Multi-Script Text Detection

arXiv 2507.20188 ↗First posted 27 Jul 2025cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

Abstract (arXiv)

Detecting text in natural scenes remains challenging, particularly for diverse scripts and arbitrarily shaped instances where visual cues alone are often insufficient. Existing methods do not fully leverage semantic context. This paper introduces SAViL-Det, a novel semantic-aware vision-language model that enhances multi-script text detection by effectively integrating textual prompts with visual features. SAViL-Det utilizes a pre-trained CLIP model combined with an Asymptotic Feature Pyramid Network (AFPN) for multi-scale visual feature fusion. The core of the proposed framework is a novel language-vision decoder that adaptively propagates fine-grained semantic information from text prompts to visual features via cross-modal attention. Furthermore, a text-to-pixel contrastive learning mechanism explicitly aligns textual and corresponding visual pixel features. Extensive experiments on challenging benchmarks demonstrate the effectiveness of the proposed approach, achieving state-of-the-art performance with F-scores of 84.8% on the benchmark multi-lingual MLT-2019 dataset and 90.2% on the curved-text CTW1500 dataset.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 22 of its 26 references that are in the map. First posted on arXiv 27 Jul 2025.

Builds on

22 of 26 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Language Matters: A Weakly Supervised Vision-Language Pre-training Approach for Scene Text Detection and Spotting8 Mar

Other references · 21

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
  2. 2019Character Region Awareness for Text Detectioncited by 274
  3. 2017EAST: An Efficient and Accurate Scene Text Detectorcited by 248
  4. 2019Shape Robust Text Detection with Progressive Scale Expansion Networkcited by 147
  5. 2019ICDAR2019 Robust Reading Challenge on Multi-lingual Scene Text Detection and Recognition -- RRC-MLT-2019cited by 138
  6. 2018TextSnake: A Flexible Representation for Detecting Text of Arbitrary Shapescited by 114
  7. 2019Efficient and Accurate Arbitrary-Shaped Text Detection with Pixel Aggregation Networkcited by 114
  8. 2022Real-Time Scene Text Detection with Differentiable Binarization and Adaptive Scale Fusioncited by 94
  9. 2021Fourier Contour Embedding for Arbitrary-Shaped Text Detectioncited by 80
  10. 2022Text Spotting Transformerscited by 70
  11. 2019Look More Than Once: An Accurate Detector for Text of Arbitrary Shapescited by 66
  12. 2018PixelLink: Detecting Scene Text via Instance Segmentationcited by 60
9 more
  1. 2022DPText-DETR: Towards Better Scene Text Detection with Dynamic Points in Transformercited by 50
  2. 2023Turning a CLIP Model into a Scene Text Detectorcited by 44
  3. 2022Vision-Language Pre-Training for Boosting Scene Text Detectorscited by 18
  4. 2023AFPN: Asymptotic Feature Pyramid Network for Object Detectioncited by 17
  5. 2021FAST: Faster Arbitrarily-Shaped Text Detector with Minimalist Kernel Representationcited by 9
  6. 2020Text Detection and Recognition in the Wild: A Reviewcited by 7
  7. 2023DeepSolo++: Let Transformer Decoder with Explicit Points Solo for Multilingual Text Spottingcited by 7
  8. 2023SRFormer: Text Detection Transformer with Incorporated Segmentation and Regressioncited by 5
  9. 2023MixNet: Toward Accurate Detection of Challenging Scene Text in the Wildcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Visual Text Processing: A Comprehensive Review and Unified Evaluation
  2. 2026MonkeyOCRv2: A Visual-Text Foundation Model for Document AI
  3. 2026All-in-One Multilingual Scene Text Recognition with Script-aware Mixture-of-Experts
  4. 2026MEVL-STP: Multi-Encoder and Vision Language Model for Arbitrarily Shaped Scene Text Spotting
  5. 2024From Training-Free to Adaptive: Empirical Insights into MLLMs' Understanding of Detection Information
  6. 2023Few-shot Domain-Adaptive Visually-fused Event Detection from Text
  7. 2024Leopard: A Vision Language Model For Text-Rich Multi-Image Tasks
  8. 2025Beyond CNNs: Efficient Fine-Tuning of Multi-Modal LLMs for Object Detection on Low-Data Regimes