Why do CNNs excel at feature extraction? A mathematical explanation
Not on the map. Why?
Abstract (arXiv)
Over the past decade deep learning has revolutionized the field of computer vision, with convolutional neural network models proving to be very effective for image classification benchmarks. However, a fundamental theoretical questions remain answered: why can they solve discrete image classification tasks that involve feature extraction? We address this question in this paper by introducing a novel mathematical model for image classification, based on feature extraction, that can be used to generate images resembling real-world datasets. We show that convolutional neural network classifiers can solve these image classification tasks with zero error. In our proof, we construct piecewise linear functions that detect the presence of features, and show that they can be realized by a convolutional network.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 20 of its 29 references that are in the map. First posted on arXiv 3 Jul 2023.
Builds on
20 of 29 references.
References · 20
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
- 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
- 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
- 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
- 2014On the Number of Linear Regions of Deep Neural Networkscited by 381
- 2017The Expressive Power of Neural Networks: A View from the Widthcited by 320
- 2016On the Expressive Power of Deep Neural Networkscited by 306
- 2016Exponential expressivity in deep neural networks through transient chaoscited by 261
- 2018Universality of Deep Convolutional Neural Networkscited by 176
- 2019Complexity of Linear Regions in Deep Networkscited by 167
8 more
- 2017Bounding and Counting Linear Regions of Deep Neural Networkscited by 161
- 2019Deep ReLU Networks Have Surprisingly Few Activation Patternscited by 161
- 2017Approximating Continuous Functions by ReLU Nets of Minimal Widthcited by 122
- 2017Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activationscited by 104
- 2013On the number of response regions of deep feed forward networks with piece-wise linear activationscited by 94
- 2015Representation Benefits of Deep Feedforward Networkscited by 86
- 2019Theoretical Issues in Deep Networks: Approximation, Optimization and Generalizationcited by 72
- 2020On the Number of Linear Regions of Convolutional Neural Networkscited by 49
Built on by
Citing papers per year: 2025: 1.
1 paper.