# Arguments for the Unsuitability of Convolutional Neural Networks for Non--Local Tasks

arXiv: https://arxiv.org/abs/2102.11944 · HTML: https://mlsanity.com/paper/2102.11944/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 15 references that are in the map.
- First posted on arXiv 23 Feb 2021.

## Builds on (most cited first, 5 of 5)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [25 years of CNNs: Can we compare to human abstraction capabilities?](https://mlsanity.com/paper/1607.08366/) (2016)
- [Five Points to Check when Comparing Visual Perception in Humans and Machines](https://mlsanity.com/paper/2004.09406/) (2020)
- [Same-different problems strain convolutional neural networks](https://mlsanity.com/paper/1802.03390/) (2018)

## Built on by (most cited first, 2 of 2)

- [Is current research on adversarial robustness addressing the right problem?](https://mlsanity.com/paper/2208.00539/) (2022)
- [Visual Reasoning in Object-Centric Deep Neural Networks: A Comparative Cognition Approach](https://mlsanity.com/paper/2402.12675/) (2024)

## Abstract (arXiv)

Convolutional neural networks have established themselves over the past years as the state of the art method for image classification, and for many datasets, they even surpass humans in categorizing images. Unfortunately, the same architectures perform much worse when they have to compare parts of an image to each other to correctly classify this image. Until now, no well-formed theoretical argument has been presented to explain this deficiency. In this paper, we will argue that convolutional layers are of little use for such problems, since comparison tasks are global by nature, but convolutional layers are local by design. We will use this insight to reformulate a comparison task into a sorting task and use findings on sorting networks to propose a lower bound for the number of parameters a neural network needs to solve comparison tasks in a generalizable way. We will use this lower bound to argue that attention, as well as iterative/recurrent processing, is needed to prevent a combinatorial explosion.