# How Deep is the Feature Analysis underlying Rapid Visual Categorization?

arXiv: https://arxiv.org/abs/1606.01167 · HTML: https://mlsanity.com/paper/1606.01167/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jun 2016.

## Built on by (most cited first, 7 of 7)

- [Harmonizing the object recognition strategies of deep neural networks with humans](https://mlsanity.com/paper/2211.04533/) (2022)
- [Performance-optimized deep neural networks are evolving into worse models of inferotemporal visual cortex](https://mlsanity.com/paper/2306.03779/) (2023)
- [Tracking Without Re-recognition in Humans and Machines](https://mlsanity.com/paper/2105.13351/) (2021)
- [Measuring Human Perception to Improve Open Set Recognition](https://mlsanity.com/paper/2209.03519/) (2022)
- [Tracking objects that change in appearance with phase synchrony](https://mlsanity.com/paper/2410.02094/) (2024)
- [RTify: Aligning Deep Neural Networks with Human Behavioral Decisions](https://mlsanity.com/paper/2411.03630/) (2024)
- [Psychophysical-Score: A Behavioral Measure for Assessing the Biological Plausibility of Visual Recognition Models](https://mlsanity.com/paper/2210.08632/) (2022)

## Abstract (arXiv)

Rapid categorization paradigms have a long history in experimental psychology: Characterized by short presentation times and speedy behavioral responses, these tasks highlight the efficiency with which our visual system processes natural object categories. Previous studies have shown that feed-forward hierarchical models of the visual cortex provide a good fit to human visual decisions. At the same time, recent work in computer vision has demonstrated significant gains in object recognition accuracy with increasingly deep hierarchical architectures. But it is unclear how well these models account for human visual decisions and what they may reveal about the underlying brain processes. We have conducted a large-scale psychophysics study to assess the correlation between computational models and human participants on a rapid animal vs. non-animal categorization task. We considered visual representations of varying complexity by analyzing the output of different stages of processing in three state-of-the-art deep networks. We found that recognition accuracy increases with higher stages of visual processing (higher level stages indeed outperforming human participants on the same task) but that human decisions agree best with predictions from intermediate stages. Overall, these results suggest that human participants may rely on visual features of intermediate complexity and that the complexity of visual representations afforded by modern deep network models may exceed those used by human participants during rapid categorization.