mlsanityData to 14 Oct 2026

When MLLMs miss what humans read: diagnosing and learning to read squint text

arXiv 2610.13465 ↗First posted 9 Oct 2026cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning also fits: Chart understanding and anomaly detection

Abstract (arXiv)

Multimodal LLMs can fail to read text embedded in an image's coarse structure (text a human recovers by squinting) and report that no text is present. We name this class Squint Text and study it as a recognition deficiency, motivated by cases from a production moderation pipeline. We introduce SquintBench, comprising 1076 production-derived moderation images, and SquintCtrl, a controlled synthetic suite for mechanism diagnosis. Frozen-weight analysis shows that hidden text leaves a coarse signal while glyph identity is weakly represented; identity becomes substantially more accessible when the correct spatial regions are supplied, implicating spatial selection as a major bottleneck. Suppressing high-frequency content also improves reading at a fixed visual-token budget, while visual token count may also contribute. These findings motivate Squint OPSD: a frozen teacher receives a downscaled view and the ground-truth transcription as privileged context, and its token-level distributions supervise a student along on-policy trajectories. At inference, the student reads the original image alone. Squint OPSD reaches 88.61 ANLS on SquintBench, versus 26.50 for the unadapted backbone, 24.44 for the best general-purpose model on the original view, and 68.60 at best for an off-the-shelf model under a downscaled view. These results substantially narrow the gap between available visual evidence and its generated read-out without auxiliary views at inference.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 38 of its 55 references that are in the map. First posted on arXiv 9 Oct 2026.

Builds on

38 of 55 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025VACoT: Rethinking Visual Data Augmentation with VLMs2 Dec
  2. 2025SemVink: Advancing VLMs' Semantic Understanding of Optical Illusions via Visual Global Thinking3 Jun
  3. 2024Towards Interpreting Visual Information Processing in Vision-Language Models9 Oct

Other references · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  6. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  7. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  8. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  9. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  10. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
  11. 2023Vision Transformers Need Registerscited by 871
  12. 2016A Diagram Is Worth A Dozen Imagescited by 804
23 more
  1. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  2. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 660
  3. 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 490
  4. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  5. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
  6. 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelscited by 455
  7. 2020TextCaps: a Dataset for Image Captioning with Reading Comprehensioncited by 400
  8. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 349
  9. 2019Scene Text Visual Question Answeringcited by 331
  10. 2023JourneyDB: A Benchmark for Generative Image Understandingcited by 205
  11. 2024LLaVA-UHD: an LMM Perceiving Any Aspect Ratio and High-Resolution Imagescited by 195
  12. 2025MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMscited by 176
  13. 2020A Theory of Usable Information Under Computational Constraintscited by 162
  14. 2024SEED-Bench-2-Plus: Benchmarking Multimodal Large Language Models with Text-Rich Visual Comprehensioncited by 146
  15. 2017ICDAR2017 Competition on Reading Chinese Text in the Wild (RCTW-17)cited by 105
  16. 2026Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoningcited by 16
  17. 2024Moderating Illicit Online Image Promotion for Unsafe User-Generated Content Games Using Large Vision-Language Modelscited by 14
  18. 2024Illusory VQA: Benchmarking and Enhancing Multimodal Models on Visual Illusionscited by 11
  19. 2024ICM-Assistant: Instruction-tuning Multimodal Large Language Models for Rule-based Explainable Image Content Moderationcited by 11
  20. 2025Multimodal Large Language Models for Text-rich Image Understanding: A Comprehensive Reviewcited by 9
  21. 2024TRINS: Towards Multimodal Language Models that Can Readcited by 8
  22. 2026Reading, Not Thinking: Understanding and Bridging the Modality Gap When Text Becomes Pixels in Multimodal LLMscited by 7
  23. 2026Global-Local Dual Perception for MLLMs in High-Resolution Text-Rich Image Translationcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025See the Text: From Tokenization to Visual Reading
  2. 2026Can Vision-Language Models See Squares? Text-Recognition Mediates Spatial Reasoning Across Three Model Families
  3. 2026Reading neq Seeing: Diagnosing and Closing the Typography Gap in Vision-Language Models
  4. 2025Text Speaks Louder than Vision: ASCII Art Reveals Textual Biases in Vision-Language Models
  5. 2025ReadBench: Measuring the Dense Text Visual Reading Ability of Vision-Language Models
  6. 2026MLLMs Get It Right, Then Get It Wrong: Tracing and Correcting Late-Layer Textual Bias
  7. 2025Words or Vision: Do Vision-Language Models Have Blind Faith in Text?
  8. 2025Mixed Signals: Decoding VLMs' Reasoning and Underlying Bias in Vision-Language Conflict