mlsanityData to 7 Oct 2026

Sorry Robot, Happy Human: Vision-Language Models Read Only One of Two Legible Typographic Layers

arXiv 2609.31403 ↗First posted 25 Sep 2026cs.CL cs.AI cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG also fits: VLM benchmarks and visual reasoning

Abstract (arXiv)

Vision-language models (VLMs), despite their success in optical character recognition (OCR) tasks, are vulnerable to typographic attacks and have a fragile structure for images with multiple text layers. In this study, the DecoyBench dataset was created using the Decoy Font method. The dataset consists of 300 images, each containing text with sharp contour lines superimposed on another text with soft shading. Six recent closed-source models from three different model families were evaluated using this dataset under two different prompting conditions (naive and guided) and at two different resolutions (512×512 and 64×64). A validation study showed that human participants could read both text layers with high accuracy. In contrast, the models, with most variants and both prompting methods, read the contour text with near-human accuracy at high resolution, but almost never fully extracted the shading text. At low resolution, the contour text could not be read by either the models or humans, while the shading text could be extracted with high accuracy. The findings indicate that the evaluated VLMs exhibit a consistent behavioral limitation when processing typographic structures containing multiple spatial frequency layers.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 6 of its 14 references that are in the map. First posted on arXiv 25 Sep 2026.

Builds on

6 of 14 references.

References · 6

  1. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  2. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  3. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 368
  4. 2024Vision-LLMs Can Fool Themselves with Self-Generated Typographic Attackscited by 36
  5. 2025SCAM: A Real-World Typographic Robustness Evaluation for Multimodal Foundation Modelscited by 12
  6. 2025Invisible Injections: Exploiting Vision-Language Models Through Steganographic Prompt Embeddingcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Texture or Semantics? Vision-Language Models Get Lost in Font Recognition
  2. 2026Reading neq Seeing: Diagnosing and Closing the Typography Gap in Vision-Language Models
  3. 2025Read or Ignore? A Unified Benchmark for Typographic-Attack Robustness and Text Recognition in Vision-Language Models
  4. 2026Reading or Guessing? Visual Grounding Failures of Vision-Language Models for OCR in Ancient Greek Editions
  5. 2025Benchmarking Vision-Language Models on Optical Character Recognition in Dynamic Video Environments
  6. 2026GlotOCR Bench: OCR Models Still Struggle Beyond a Handful of Unicode Scripts
  7. 2024TAP-VL: Text Layout-Aware Pre-training for Enriched Vision-Language Models
  8. 2026Text is All You Need for Vision-Language Model Jailbreaking