Similarities and differences between stimulus tuning in the inferotemporal visual cortex and convolutional networks

Deep convolutional neural networks (CNNs) trained for object classification have a number of striking similarities with the primate ventral visual stream. In particular, activity in early, intermediate, and late layers is closely related to activity in V1, V4, and the inferotemporal cortex (IT). This study further compares activity in late layers of object-classification CNNs to activity patterns reported in the IT electrophysiology literature. There are a number of close similarities, including the distributions of population response sparseness across stimuli, and the distribution of size tuning bandwidth. Statisics of scale invariance, responses to clutter and occlusion, and orientation tuning are less similar. Statistics of object selectivity are quite different. These results agree with recent studies that highlight strong parallels between object-categorization CNNs and the ventral stream, and also highlight differences that could perhaps be reduced in future CNNs.