mlsanityData to 7 Oct 2026

One Patch Is Enough: Reinforcement-Optimized Visual Token Grounding for MLLM-Based Scene Text Spotting

arXiv 2607.27902 ↗First posted 30 Jul 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Scene text spotting requires high-precision alignment between textual recognition and spatial localization. While visual-token grounding has emerged as a promising formulation for Multimodal Large Language Models (MLLMs), the previous multi-patch paradigm often introduces redundant noise and localization ambiguity, particularly for dense or small text instances. To address this, we propose Single-Patch Text Spotting (SPaTS), a vision-centric framework that routes each text instance through a single anchor visual token and then recovers geometry via full-image refinement. To accurately identify this anchor without oracle labels, we introduce Single-Patch Selective Optimization (SPaSO), a reinforcement learning framework that optimizes discrete visual-token selection using patch-level rewards. To further improve representation robustness and localization precision, we introduce Directional Embedding Alignment (DEA) to suppress unstable norm bias by decoupling feature magnitude and direction, and Patch-Enhanced Decoding (PED) to fuse the routed anchor with language semantics and cross-attend over the full-image feature map for geometry-aware boundary regression beyond coordinate-space surrogates. Extensive experiments demonstrate that SPaTS consistently and significantly outperforms both frontier closed-source MLLMs and OCR MLLMs. Code is available at https://github.com/eeNickTang/SPaTS.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 54 references that are in the map. First posted on arXiv 30 Jul 2026.

Builds on

23 of 54 references.

References · 23

  1. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  2. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  3. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,075
  4. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 501
  5. 2023Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Modelscited by 356
  6. 2016Rationalizing Neural Predictionscited by 281
  7. 2019Mask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapescited by 171
  8. 2020ABCNet: Real-time Scene Text Spotting with Adaptive Bezier-Curve Networkcited by 170
  9. 2025LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Tokencited by 164
  10. 2017Total-Text: A Comprehensive Dataset for Scene Text Detection and Recognitioncited by 161
  11. 2023Osprey: Pixel Understanding with Visual Instruction Tuningcited by 158
  12. 2018Scene Text Detection and Recognition: The Deep Learning Eracited by 114
11 more
  1. 2018FOTS: Fast Oriented Text Spotting with a Unified Networkcited by 109
  2. 2022Real-Time Scene Text Detection with Differentiable Binarization and Adaptive Scale Fusioncited by 94
  3. 2021ABCNet v2: Adaptive Bezier-Curve Network for Real-time End-to-end Text Spottingcited by 84
  4. 2022DeepSolo: Let Transformer Decoder with Explicit Points Solo for Text Spottingcited by 74
  5. 2023SPTS v2: Single-Point Scene Text Spottingcited by 66
  6. 2024VLM-RL: A Unified Vision Language Models and Reinforcement Learning Framework for Safe Autonomous Drivingcited by 49
  7. 2025Patch-as-Decodable-Token: Towards Unified Multi-Modal Vision Tasks in MLLMscited by 20
  8. 2026Multimodal OCR: Parse Anything from Documentscited by 12
  9. 2025OCRGenBench: A Comprehensive Benchmark for Evaluating OCR Generative Capabilitiescited by 8
  10. 2024ClawMachine: Learning to Fetch Visual Tokens for Referential Comprehensioncited by 6
  11. 2025Smaller But Better: Unifying Layout Generation with Smaller Large Language Modelscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SupGRPO: Enhancing GRPO with Matching-based Online SFT for Text Spotting
  2. 2026Layer-Aware Position Embeddings for Visual Token Pruning in Multimodal Large Language Models
  3. 2025FlashVLM: Text-Guided Visual Token Selection for Large Multimodal Models
  4. 2025Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Grounding
  5. 2025REVEALER: Reinforcement-Guided Visual Reasoning for Element-Level Text-Image Alignment Evaluation
  6. 2026MEVL-STP: Multi-Encoder and Vision Language Model for Arbitrarily Shaped Scene Text Spotting
  7. 2026One Layer's Trash is Another Layer's Treasure: Adaptive Layer-wise Visual Token Selection in LVLMs
  8. 2026TIPSv2: Advancing Vision-Language Pretraining with Enhanced Patch-Text Alignment