# Disentangling the Factors of Convergence between Brains and Computer Vision Models

arXiv: https://arxiv.org/abs/2508.18226 · HTML: https://mlsanity.com/paper/2508.18226/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 56 references that are in the map.
- First posted on arXiv 25 Aug 2025.

## Builds on (most cited first, 13 of 13)

- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025)
- [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://mlsanity.com/paper/2502.14786/) (2025)
- [The Platonic Representation Hypothesis](https://mlsanity.com/paper/2405.07987/) (2024)
- [How Deep Neural Networks Learn Compositional Data: The Random Hierarchy Model](https://mlsanity.com/paper/2307.02129/) (2023)
- [Toward a realistic model of speech processing in the brain with self-supervised learning](https://mlsanity.com/paper/2206.01685/) (2022)
- [Dimensions underlying the representational alignment of deep neural networks with humans](https://mlsanity.com/paper/2406.19087/) (2024)
- [Scaling laws for decoding images from brain activity](https://mlsanity.com/paper/2501.15322/) (2025)
- [When Representations Align: Universality in Representation Learning Dynamics](https://mlsanity.com/paper/2402.09142/) (2024)
- [Alignment between Brains and AI: Evidence for Convergent Evolution across Modalities, Scales and Training Trajectories](https://mlsanity.com/paper/2507.01966/) (2025)
- [In silico discovery of representational relationships across visual cortex](https://mlsanity.com/paper/2411.10872/) (2024)
- [Brain-like emergent properties in deep networks: impact of network architecture, datasets and training](https://mlsanity.com/paper/2411.16326/) (2024)

## Built on by (most cited first, 13 of 13)

- [Revisiting the Platonic Representation Hypothesis: An Aristotelian View](https://mlsanity.com/paper/2602.14486/) (2026)
- [Representation Potentials of Foundation Models for Multimodal Alignment: A Survey](https://mlsanity.com/paper/2510.05184/) (2025)
- [Self-Distillation of Hidden Layers for Self-Supervised Representation Learning](https://mlsanity.com/paper/2603.15553/) (2026)
- [Local Intrinsic Dimension of Representations Predicts Alignment and Generalization in AI Models and Human Brain](https://mlsanity.com/paper/2601.22722/) (2026)
- [Toward Aristotelian Medical Representations: Backpropagation-Free Layer-wise Analysis for Interpretable Generalized Metric Learning on MedMNIST](https://mlsanity.com/paper/2604.06017/) (2026)
- [CanViT: Toward Active-Vision Foundation Models](https://mlsanity.com/paper/2603.22570/) (2026)
- [Representation Alignment Rests on Linear Structure](https://mlsanity.com/paper/2605.28870/) (2026)
- [A Unified Geometric Space for Topological Alignment Between Transformer-Based Models and Human Brain Networks](https://mlsanity.com/paper/2510.24342/) (2025)
- [Left-right asymmetry in predicting brain activity from LLMs' representations emerges with their formal linguistic competence](https://mlsanity.com/paper/2602.12811/) (2026)
- [Style-Aware Gloss Control for Generative Non-Photorealistic Rendering](https://mlsanity.com/paper/2602.16611/) (2026)
- [Misalignment Between Backpropagation and the Hierarchy of Brain Responses to Images](https://mlsanity.com/paper/2605.28693/) (2026)
- [What Makes Linguistic Representations Good Models of High-Level Visual Perception in the Human Brain?](https://mlsanity.com/paper/2607.16214/) (2026)
- [IRIS: A Visual Cortex-Inspired Framework for Analyzing Orientation Selectivity in Vision Transformers](https://mlsanity.com/paper/2608.05122/) (2026)

## Abstract (arXiv)

Many AI models trained on natural images develop representations that resemble those of the human brain. However, the factors that drive this brain-model similarity remain poorly understood. To disentangle how the model, training and data independently lead a neural network to develop brain-like representations, we trained a family of self-supervised vision transformers (DINOv3) that systematically varied these different factors. We compare their representations of images to those of the human brain recorded with both fMRI and MEG, providing high resolution in spatial and temporal analyses. We assess the brain-model similarity with three complementary metrics focusing on overall representational similarity, topographical organization, and temporal dynamics. We show that all three factors - model size, training amount, and image type - independently and interactively impact each of these brain similarity metrics. In particular, the largest DINOv3 models trained with the most human-centric images reach the highest brain-similarity. This emergence of brain-like representations in AI models follows a specific chronology during training: models first align with the early representations of the sensory cortices, and only align with the late and prefrontal representations of the brain with considerably more training. Finally, this developmental trajectory is indexed by both structural and functional properties of the human cortex: the representations that are acquired last by the models specifically align with the cortical areas with the largest developmental expansion, thickness, least myelination, and slowest timescales. Overall, these findings disentangle the interplay between architecture and experience in shaping how artificial neural networks come to see the world as humans do, thus offering a promising framework to understand how the human brain comes to represent its visual world.