mlsanityData to 14 Oct 2026

Visible Yet Unreadable: A Systematic Blind Spot of Vision Language Models Across Writing Systems

arXiv 2509.06996 ↗First posted 4 Sep 2025cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Writing is a universal cultural technology that reuses vision for symbolic communication. Humans display striking resilience: we readily recognize words even when characters are fragmented, fused, or partially occluded. This paper investigates whether advanced vision language models (VLMs) share this resilience. We construct two psychophysics inspired benchmarks across distinct writing systems, Chinese logographs and English alphabetic words, by splicing, recombining, and overlaying glyphs to yield ''visible but unreadable'' stimuli for models while remaining legible to humans. Despite strong performance on clean text, contemporary VLMs show a severe drop under these perturbations, frequently producing unrelated or incoherent outputs. The pattern suggests a structural limitation: models heavily leverage generic visual invariances but under rely on compositional priors needed for robust literacy. We release stimuli generation code, prompts, and evaluation protocols to facilitate transparent replication and follow up work. Our findings motivate architectures and training strategies that encode symbol segmentation, composition, and binding across scripts, and they delineate concrete challenges for deploying multimodal systems in education, accessibility, cultural heritage, and security.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 13 of its 34 references that are in the map. First posted on arXiv 4 Sep 2025.

Builds on

13 of 34 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Flamingo: a Visual Language Model for Few-Shot Learning29 Apr

Other references · 12

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  6. 2024GPT-4o System Cardcited by 4,588
  7. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  8. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
  9. 2018Relational inductive biases, deep learning, and graph networkscited by 1,360
  10. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  11. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 290
  12. 2024The Impact of Visual Information in Chinese Characters: Evaluating Large Models' Ability to Recognize and Utilize Radicalscited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Text Speaks Louder than Vision: ASCII Art Reveals Textual Biases in Vision-Language Models
  2. 2025See the Text: From Tokenization to Visual Reading
  3. 2025Vision Language Models Map Logos to Text via Semantic Entanglement in the Visual Projector
  4. 2025Words or Vision: Do Vision-Language Models Have Blind Faith in Text?
  5. 2025Toward Socially Aware Vision-Language Models: Evaluating Cultural Competence Through Multimodal Story Generation
  6. 2026Can Vision Replace Text in Working Memory? Evidence from Spatial n-Back in Vision-Language Models
  7. 2025Emergence of Text Readability in Vision Language Models
  8. 2025The Mechanistic Emergence of Symbol Grounding in Language Models