Lesioned Multimodal Language Models Reproduce Aphasic Picture-Naming Patterns
On the map: Multimodal LLMs / vision-language models → Medical vision-language models
Abstract (arXiv)
Aphasia following stroke commonly produces systematic naming errors with characteristic profiles, but whether general-purpose language models not designed for clinical simulation can reproduce these patterns remains untested. We investigated (1) whether lesions or controlled perturbations to a multimodal language model can reproduce different types of errors in picture naming, and (2) whether the framework can reproduce the complete error profile of individual persons with aphasia (PWAs). Using LLaVA 1.6, we evaluated perturbation configurations that varied the layer, proportion, and amount of noise applied to model units. We examined 278 PWAs on the Philadelphia Naming Test, classifying responses into seven categories using a validated neural classifier. Six of seven response categories (correct, semantic, mixed, unrelated, neologism, no response errors) emerged at clinically-comparable proportions across distinct parameter space regions, with formal paraphasia being the exception. Searching the perturbation space revealed configurations that reproduced the individual error profile in at least six of seven categories for 97.8% of PWAs and in all seven categories for 79.5% of PWAs. Monte Carlo baselines confirmed that this matching reflects joint inter-category structure rather than marginal overlap. These results establish a quantitative framework for reproducing individual aphasic error patterns in picture naming. They suggest the potential for language models to serve as digital twins of individuals with post-stroke aphasia.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 9 of its 37 references that are in the map. First posted on arXiv 13 Jul 2026.
Builds on
9 of 37 references.
References · 9
- 2017Attention Is All You Needcited by 59,932
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2022Locating and Editing Factual Associations in GPTcited by 2,609
- 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
- 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
- 2026Component-Level Lesioning of Language Models Reveals Clinically Aligned Aphasia Phenotypescited by 3
- 2026Artificial Aphasias in Lesioned Language Modelscited by 2
- 2025Bridging Brains and Models: MoE-Based Functional Lesions for Simulating and Rehabilitating Aphasiacited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Do Language Models Align with Brains? Prediction Scores Are Not Enough
- 2026NeuroVLM-Bench: Evaluation of Vision-Enabled Large Language Models for Clinical Reasoning in Neurological Disorders
- 2026Prompts Without Evidence: How Neuroimaging Mentions Shift Clinical Vision-Language Model Predictions
- 2026Computational Lesions in Multilingual Language Models Separate Shared and Language-specific Brain Alignment
- 2026Confident but Unreliable: A Behavioral Safety Audit of Vision-Language Models on Brain MRI
- 2026Evaluating the Diagnostic Robustness of Vision-Language Models Under Visual and Textual Perturbations
- 2026Cognitive Expert Language Models Better Align with the Corresponding Brain Systems
- 2025Inducing Dyslexia in Vision Language Models