Extremely coarse learning objectives induce human-aligned representations in AI vision models
Not on the map. Why?
Abstract (arXiv)
Artificial neural networks trained on visual tasks develop internal representations resembling those of the primate visual system, a discovery that has guided a decade of computational neuroscience. Research on building brain-aligned models has progressively embraced finer-grained learning ob- jectives, from object classification to contrastive self-supervised objectives that maximize distinc- tions among individual images. Yet the effect of learning-signal granularity on brain alignment remains largely unexamined. Here we systematically investigate how the granularity of a learning signal shapes representational alignment with human vision. We parametrically vary the number of training classes using a data-driven approach that partitions a set of training images into differ- ent numbers of categories via PCA-based splits of pretrained embeddings. We train hundreds of neural networks across convolutional and transformer architectures on these coarse classification tasks and compare their representations with human fMRI responses, macaque electrophysiology recordings, and human behavior. We find that networks trained to distinguish as few as eight broad categories learn representations that match or exceed the neural alignment of models distinguishing 1,000 classes. Even more strikingly, these coarsely trained networks align more closely with hu- man perceptual similarity judgments than all other models evaluated, including networks trained with fine-grained supervision or self-supervision as well as leading large-scale vision models. These results demonstrate that human-like visual representations can emerge from surprisingly simple learning objectives, reframing what learning signals vision may require and opening a path toward building AI systems that are more aligned with human perception.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 24 of its 70 references that are in the map. First posted on arXiv 7 May 2026.
Builds on
24 of 70 references.
References · 24
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2022A ConvNet for the 2020scited by 3,545
- 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 322
- 2019Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNscited by 90
- 2022Harmonizing the object recognition strategies of deep neural networks with humanscited by 59
- 2014Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognitioncited by 50
- 2022The neuroconnectionist research programmecited by 47
- 2023Improving neural network representations using human similarity judgmentscited by 38
- 2023Performance-optimized deep neural networks are evolving into worse models of inferotemporal visual cortexcited by 18
12 more
- 2023On the Trade-off of Intra-/Inter-class Diversity for Supervised Pre-trainingcited by 14
- 2019Understanding the Impact of Label Granularity on CNN-based Image Classificationcited by 13
- 2021Statistical inference on representational geometriescited by 13
- 2024Universal dimensions of visual representationcited by 12
- 2024Scaling Laws for Task-Optimized Models of the Primate Visual Ventral Streamcited by 12
- 2020Weakly Supervised Representation Learning with Coarse Labelscited by 11
- 2021Semi-Supervised Learning with Taxonomic Labelscited by 11
- 2021Coarse-to-Fine Curriculum Learningcited by 9
- 2018Learning Hierarchical Visual Representations in Deep Neural Networks Using Hierarchical Linguistic Labelscited by 4
- 2015Basic Level Categorization Facilitates Visual Object Recognitioncited by 3
- 2024Vision CNNs trained to estimate spatial latents learned similar ventral-stream-aligned representationscited by 2
- 2025The Advantage of Fine-Grained Trainingcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.