mlsanityData to 14 Oct 2026

Emergence of Text Readability in Vision Language Models

arXiv 2506.19389 ↗First posted 24 Jun 2025cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

We investigate how the ability to recognize textual content within images emerges during the training of Vision-Language Models (VLMs). Our analysis reveals a critical phenomenon: the ability to read textual information in a given image (text readability) emerges abruptly after substantial training iterations, in contrast to semantic content understanding which develops gradually from the early stages of training. This delayed emergence may reflect how contrastive learning tends to initially prioritize general semantic understanding, with text-specific symbolic processing developing later. Interestingly, the ability to match images with rendered text develops even slower, indicating a deeper need for semantic integration. These findings highlight the need for tailored training strategies to accelerate robust text comprehension in VLMs, laying the groundwork for future research on optimizing multimodal learning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 30 of its 32 references that are in the map. First posted on arXiv 24 Jun 2025.

Builds on

30 of 32 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks21 Dec
  2. 2023UReader: Universal OCR-free Visually-situated Language Understanding with Multimodal Large Language Model8 Oct

Other references · 28

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  6. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  7. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  8. 2022A ConvNet for the 2020scited by 3,579
  9. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  10. 2022Emergent Abilities of Large Language Modelscited by 2,512
  11. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  12. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 766
16 more
  1. 2023DataComp: In search of the next generation of multimodal datasetscited by 644
  2. 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 537
  3. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 523
  4. 2018nocaps: novel object captioning at scalecited by 444
  5. 2023Data Filtering Networkscited by 250
  6. 2023What does CLIP know about a red circle? Visual prompt engineering for VLMscited by 218
  7. 2023Interpreting CLIP's Image Representation via Text-Based Decompositioncited by 196
  8. 2023Hyperbolic Image-Text Representationscited by 136
  9. 2024Understanding Emergent Abilities of Language Models from the Loss Perspectivecited by 94
  10. 2022Disentangling visual and written concepts in CLIPcited by 58
  11. 2022CLIPPO: Image-and-Language Understanding from Pixels Onlycited by 55
  12. 2023Parrot Captions Teach CLIP to Spot Textcited by 17
  13. 2024Probabilistic Language-Image Pre-Trainingcited by 16
  14. 2024Hierarchical Visual Feature Aggregation for OCR-Free Document Understandingcited by 12
  15. 2024Predicting Emergent Capabilities by Finetuningcited by 10
  16. 2023Multi-Modal Representation Learning with Text-Driven Soft Maskscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024The Narrow Gate: Localized Image-Text Communication in Native Multimodal Models
  2. 2025Words or Vision: Do Vision-Language Models Have Blind Faith in Text?
  3. 2025Perceiving Beyond Language Priors: Enhancing Visual Comprehension and Attention in Multimodal Models
  4. 2026VISTA-Bench: Do Vision-Language Models Really Understand Visualized Text as Well as Pure Text?
  5. 2024From Training-Free to Adaptive: Empirical Insights into MLLMs' Understanding of Detection Information
  6. 2025ReadBench: Measuring the Dense Text Visual Reading Ability of Vision-Language Models
  7. 2025Text Speaks Louder than Vision: ASCII Art Reveals Textual Biases in Vision-Language Models
  8. 2025Rethinking the Text-Vision Reasoning Imbalance in MLLMs through the Lens of Training Recipes