mlsanityData to 14 Oct 2026

Language learning shapes visual category-selectivity in deep neural networks

arXiv 2502.16456 ↗First posted 23 Feb 2025q-bio.NC cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Category-selective regions in the human brain-such as the fusiform face area (FFA), extrastriate body area (EBA), parahippocampal place area (PPA), and visual word form area (VWFA)-support high-level visual recognition. Here, we investigate whether artificial neural networks (ANNs) exhibit analogous category-selective neurons and how these representations are shaped by language experience. Using an fMRI-inspired functional localizer approach, we identified face-, body-, place-, and word-selective neurons in deep networks presented with category images and scrambled controls. Both the purely visual ResNet and a linguistically supervised Lang-Learned ResNet contained category-selective neurons that increased in proportion across layers. However, compared to the vision-only model, the Lang-Learned ResNet showed a greater number but lower specificity of category-selective neurons, along with reduced spatial localization and attenuated activation strength-indicating a shift toward more distributed, semantically aligned coding. These effects were replicated in the large-scale vision-language model CLIP. Together, our findings reveal that language experience systematically reorganizes visual category representations in ANNs, providing a computational parallel to how linguistic context may shape categorical organization in the human brain.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 54 references that are in the map. First posted on arXiv 23 Feb 2025.

Builds on

13 of 54 references.

References · 13

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021Understanding Robustness of Transformers for Image Classificationcited by 249
  3. 2021Partial success in closing the gap between human and machine visioncited by 185
  4. 2014Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Brain's Ventral Visual Pathwaycited by 94
  5. 2019Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNscited by 90
  6. 2019Recurrence is required to capture the representational dynamics of the human visual systemcited by 58
  7. 2014Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognitioncited by 50
  8. 2021Modeling Category-Selective Cortical Regions with Topographic Variational Autoencoderscited by 10
  9. 2023End-to-end topographic networks as models of cortical map formation and human visual behaviour: moving beyond convolutionscited by 8
  10. 2024Achieving More Human Brain-Like Vision via Human EEG Representational Alignmentcited by 4
  11. 2023Generate your neural signals from mine: individual-to-individual EEG converterscited by 3
  12. 2024Teaching CORnet Human fMRI Representations for Enhanced Model-Brain Alignmentcited by 2
1 more
  1. 2024Cracking the neural code for word recognition in convolutional neural networkscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Vision-Language Models Align with Human Neural Representations in Concept Processing
  2. 2025Uncovering Brain-Like Hierarchical Patterns in Vision-Language Models through fMRI-Based Neural Encoding
  3. 2025Uncovering Semantic Selectivity of Latent Groups in Higher Visual Cortex with Mutual Information-Guided Diffusion
  4. 2025NeuroVolve: Evolving Visual Stimuli toward Programmable Neural Objectives
  5. 2025Language-Specific Layer Matters: Efficient Multilingual Enhancement for Large Vision-Language Models
  6. 2026Linguistic Context Recodes Visual Representations in Vision-Language Models
  7. 2024The LLM Language Network: A Neuroscientific Approach for Identifying Causally Task-Relevant Units
  8. 2026Neural networks are more modular than single neurons suggest