mlsanityData to 14 Oct 2026

Inducing Dyslexia in Vision Language Models

arXiv 2509.24597 ↗First posted 29 Sep 2025cs.CL cs.LGcited by 2

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Dyslexia, a neurodevelopmental disorder characterized by persistent reading difficulties, is often linked to reduced activity of the visual word form area (VWFA) in the ventral occipito-temporal cortex. Traditional approaches to studying dyslexia, such as behavioral and neuroimaging methods, have provided valuable insights but remain limited in their ability to test causal hypotheses about the underlying mechanisms of reading impairments. In this study, we use large-scale vision-language models (VLMs) to simulate dyslexia by functionally identifying and perturbing artificial analogues of word processing. Using stimuli from cognitive neuroscience, we identify visual-word-form-selective units within VLMs and demonstrate that they predict human VWFA neural responses. Ablating model VWF units leads to selective impairments in reading tasks while general visual and language comprehension abilities remain intact. In particular, the resulting model matches dyslexic humans' phonological deficits without a significant change in orthographic processing, and mirrors dyslexic behavior in font sensitivity. Taken together, our modeling results replicate key characteristics of dyslexia and establish a computational framework for investigating brain disorders.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 12 of its 69 references that are in the map. First posted on arXiv 29 Sep 2025.

Builds on

12 of 69 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Human-like object concept representations emerge naturally in multimodal large language models1 Jul

Other references · 11

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  2. 2024Qwen2 Technical Reportcited by 2,120
  3. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 678
  4. 2019RAVEN: A Dataset for Relational and Analogical Visual rEasoNingcited by 226
  5. 2021MoEfication: Transformer Feed-forward Layers are Mixtures of Expertscited by 152
  6. 2023Instruction-tuning Aligns LLMs to the Human Braincited by 40
  7. 2024The LLM Language Network: A Neuroscientific Approach for Identifying Causally Task-Relevant Unitscited by 34
  8. 2024Scaling Laws for Task-Optimized Models of the Primate Visual Ventral Streamcited by 13
  9. 2024TopoLM: brain-like spatio-functional organization in a topographic language modelcited by 12
  10. 2025Contour Integration Underlies Human-Like Visioncited by 6
  11. 2024Handwriting Anomalies and Learning Disabilities through Recurrent Neural Networks and Geometric Pattern Analysiscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Contravariance Theory: Strong Alignment for Minimal Solutions to Hard Taskscited by 2
  2. 2026Reward Valuation in Large Language Models: Causal Induction of Anhedoniacited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Visual Language Models show widespread visual deficits on neuropsychological tests
  2. 2026Component-Level Lesioning of Language Models Reveals Clinically Aligned Aphasia Phenotypes
  3. 2025Uncovering Brain-Like Hierarchical Patterns in Vision-Language Models through fMRI-Based Neural Encoding
  4. 2024Vision-Language Models Align with Human Neural Representations in Concept Processing
  5. 2026Do VLMs Align Better with Humans than LLMs during Natural Reading?
  6. 2026Lesioned Multimodal Language Models Reproduce Aphasic Picture-Naming Patterns
  7. 2026Interpreting Brain Responses to Language with Sparse Features from Language Models
  8. 2025Human Cognitive Benchmarks Reveal Foundational Visual Gaps in MLLMs