# Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition

arXiv: https://arxiv.org/abs/1406.3284 · HTML: https://mlsanity.com/paper/1406.3284/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2014.

## Built on by (most cited first, 30 of 50)

- [Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspaces](https://mlsanity.com/paper/2212.14855/) (2022)
- [Performance-optimized deep neural networks are evolving into worse models of inferotemporal visual cortex](https://mlsanity.com/paper/2306.03779/) (2023)
- [Contributions of Shape, Texture, and Color in Visual Recognition](https://mlsanity.com/paper/2207.09510/) (2022)
- [Self-attention in Vision Transformers Performs Perceptual Grouping, Not Attention](https://mlsanity.com/paper/2303.01542/) (2023)
- [Traveling Waves Encode the Recent Past and Enhance Sequence Learning](https://mlsanity.com/paper/2309.08045/) (2023)
- [The Sensorium competition on predicting large-scale mouse primary visual cortex activity](https://mlsanity.com/paper/2206.08666/) (2022)
- [The Topology and Geometry of Neural Representations](https://mlsanity.com/paper/2309.11028/) (2023)
- [Visio-Linguistic Brain Encoding](https://mlsanity.com/paper/2204.08261/) (2022)
- [Aligning Multimodal Representations through an Information Bottleneck](https://mlsanity.com/paper/2506.04870/) (2025)
- [On the relation between statistical learning and perceptual distances](https://mlsanity.com/paper/2106.04427/) (2021)
- [Comparing object recognition in humans and deep convolutional neural networks -- An eye tracking study](https://mlsanity.com/paper/2108.00107/) (2021)
- [Better artificial intelligence does not mean better models of biology](https://mlsanity.com/paper/2504.16940/) (2025)
- [MindSet: Vision. A toolbox for testing DNNs on key psychological experiments](https://mlsanity.com/paper/2404.05290/) (2024)
- [V1T: large-scale mouse V1 response prediction using a Vision Transformer](https://mlsanity.com/paper/2302.03023/) (2023)
- [Robustified ANNs Reveal Wormholes Between Human Category Percepts](https://mlsanity.com/paper/2308.06887/) (2023)
- [Training on Foveated Images Improves Robustness to Adversarial Attacks](https://mlsanity.com/paper/2308.00854/) (2023)
- [Learning Interpretable Representations Leads to Semantically Faithful EEG-to-Text Generation](https://mlsanity.com/paper/2505.17099/) (2025)
- [Bi-Orthogonal Factor Decomposition for Vision Transformers](https://mlsanity.com/paper/2601.05328/) (2026)
- [Guiding Visual Attention in Deep Convolutional Neural Networks Based on Human Eye Movements](https://mlsanity.com/paper/2206.10587/) (2022)
- [Modeling Object Recognition in Newborn Chicks using Deep Neural Networks](https://mlsanity.com/paper/2106.07185/) (2021)
- [Functional Network: A Novel Framework for Interpretability of Deep Neural Networks](https://mlsanity.com/paper/2205.11702/) (2022)
- [Is current research on adversarial robustness addressing the right problem?](https://mlsanity.com/paper/2208.00539/) (2022)
- [A spatiotemporal style transfer algorithm for dynamic visual stimulus generation](https://mlsanity.com/paper/2403.04940/) (2024)
- [Can Transformers Smell Like Humans?](https://mlsanity.com/paper/2411.03038/) (2024)
- [Towards Neurocognitive-Inspired Intelligence: From AI's Structural Mimicry to Human-Like Functional Cognition](https://mlsanity.com/paper/2510.13826/) (2025)
- [Investigating The Functional Roles of Attention Heads in Vision Language Models: Evidence for Reasoning Modules](https://mlsanity.com/paper/2512.10300/) (2025)
- [A brain basis of dynamical intelligence for AI and computational neuroscience](https://mlsanity.com/paper/2105.07284/) (2021)
- [Modeling biological face recognition with deep convolutional neural networks](https://mlsanity.com/paper/2208.06681/) (2022)
- [Net2Brain: A Toolbox to compare artificial vision models with human brain responses](https://mlsanity.com/paper/2208.09677/) (2022)
- [A novel feature-scrambling approach reveals the capacity of convolutional neural networks to learn spatial relations](https://mlsanity.com/paper/2212.06021/) (2022)

## Abstract (arXiv)

The primate visual system achieves remarkable visual object recognition performance even in brief presentations and under changes to object exemplar, geometric transformations, and background variation (a.k.a. core visual object recognition). This remarkable performance is mediated by the representation formed in inferior temporal (IT) cortex. In parallel, recent advances in machine learning have led to ever higher performing models of object recognition using artificial deep neural networks (DNNs). It remains unclear, however, whether the representational performance of DNNs rivals that of the brain. To accurately produce such a comparison, a major difficulty has been a unifying metric that accounts for experimental limitations such as the amount of noise, the number of neural recording sites, and the number trials, and computational limitations such as the complexity of the decoding classifier and the number of classifier training examples. In this work we perform a direct comparison that corrects for these experimental limitations and computational considerations. As part of our methodology, we propose an extension of "kernel analysis" that measures the generalization accuracy as a function of representational complexity. Our evaluations show that, unlike previous bio-inspired models, the latest DNNs rival the representational performance of IT cortex on this visual object recognition task. Furthermore, we show that models that perform well on measures of representational performance also perform well on measures of representational similarity to IT and on measures of predicting individual IT multi-unit responses. Whether these DNNs rely on computational mechanisms similar to the primate visual system is yet to be determined, but, unlike all previous bio-inspired models, that possibility cannot be ruled out merely on representational performance grounds.