# SAViL-Det: Semantic-Aware Vision-Language Model for Multi-Script Text Detection

arXiv: https://arxiv.org/abs/2507.20188 · HTML: https://mlsanity.com/paper/2507.20188/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 22 of its 26 references that are in the map.
- First posted on arXiv 27 Jul 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Document understanding and multimodal RAG](https://mlsanity.com/mllm/document-understanding-and-multimodal-rag/)

## Builds on (most cited first, 22 of 22)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Character Region Awareness for Text Detection](https://mlsanity.com/paper/1904.01941/) (2019)
- [EAST: An Efficient and Accurate Scene Text Detector](https://mlsanity.com/paper/1704.03155/) (2017)
- [Shape Robust Text Detection with Progressive Scale Expansion Network](https://mlsanity.com/paper/1903.12473/) (2019)
- [ICDAR2019 Robust Reading Challenge on Multi-lingual Scene Text Detection and Recognition -- RRC-MLT-2019](https://mlsanity.com/paper/1907.00945/) (2019)
- [TextSnake: A Flexible Representation for Detecting Text of Arbitrary Shapes](https://mlsanity.com/paper/1807.01544/) (2018)
- [Efficient and Accurate Arbitrary-Shaped Text Detection with Pixel Aggregation Network](https://mlsanity.com/paper/1908.05900/) (2019)
- [Real-Time Scene Text Detection with Differentiable Binarization and Adaptive Scale Fusion](https://mlsanity.com/paper/2202.10304/) (2022)
- [Fourier Contour Embedding for Arbitrary-Shaped Text Detection](https://mlsanity.com/paper/2104.10442/) (2021)
- [Text Spotting Transformers](https://mlsanity.com/paper/2204.01918/) (2022)
- [Look More Than Once: An Accurate Detector for Text of Arbitrary Shapes](https://mlsanity.com/paper/1904.06535/) (2019)
- [PixelLink: Detecting Scene Text via Instance Segmentation](https://mlsanity.com/paper/1801.01315/) (2018)
- [DPText-DETR: Towards Better Scene Text Detection with Dynamic Points in Transformer](https://mlsanity.com/paper/2207.04491/) (2022)
- [Turning a CLIP Model into a Scene Text Detector](https://mlsanity.com/paper/2302.14338/) (2023)
- [Language Matters: A Weakly Supervised Vision-Language Pre-training Approach for Scene Text Detection and Spotting](https://mlsanity.com/paper/2203.03911/) (2022)
- [Vision-Language Pre-Training for Boosting Scene Text Detectors](https://mlsanity.com/paper/2204.13867/) (2022)
- [AFPN: Asymptotic Feature Pyramid Network for Object Detection](https://mlsanity.com/paper/2306.15988/) (2023)
- [FAST: Faster Arbitrarily-Shaped Text Detector with Minimalist Kernel Representation](https://mlsanity.com/paper/2111.02394/) (2021)
- [Text Detection and Recognition in the Wild: A Review](https://mlsanity.com/paper/2006.04305/) (2020)
- [DeepSolo++: Let Transformer Decoder with Explicit Points Solo for Multilingual Text Spotting](https://mlsanity.com/paper/2305.19957/) (2023)
- [SRFormer: Text Detection Transformer with Incorporated Segmentation and Regression](https://mlsanity.com/paper/2308.10531/) (2023)
- [MixNet: Toward Accurate Detection of Challenging Scene Text in the Wild](https://mlsanity.com/paper/2308.12817/) (2023)

## Abstract (arXiv)

Detecting text in natural scenes remains challenging, particularly for diverse scripts and arbitrarily shaped instances where visual cues alone are often insufficient. Existing methods do not fully leverage semantic context. This paper introduces SAViL-Det, a novel semantic-aware vision-language model that enhances multi-script text detection by effectively integrating textual prompts with visual features. SAViL-Det utilizes a pre-trained CLIP model combined with an Asymptotic Feature Pyramid Network (AFPN) for multi-scale visual feature fusion. The core of the proposed framework is a novel language-vision decoder that adaptively propagates fine-grained semantic information from text prompts to visual features via cross-modal attention. Furthermore, a text-to-pixel contrastive learning mechanism explicitly aligns textual and corresponding visual pixel features. Extensive experiments on challenging benchmarks demonstrate the effectiveness of the proposed approach, achieving state-of-the-art performance with F-scores of 84.8% on the benchmark multi-lingual MLT-2019 dataset and 90.2% on the curved-text CTW1500 dataset.