# A Mathematical Theory of Deep Convolutional Neural Networks for Feature Extraction

arXiv: https://arxiv.org/abs/1512.06293 · HTML: https://mlsanity.com/paper/1512.06293/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Dec 2015.

## Built on by (most cited first, 30 of 46)

- [Identifiability of latent-variable and structural-equation models: from linear to nonlinear](https://mlsanity.com/paper/2302.02672/) (2023)
- [On the Principles of Parsimony and Self-Consistency for the Emergence of Intelligence](https://mlsanity.com/paper/2207.04630/) (2022)
- [Approximation and Learning with Deep Convolutional Models: a Kernel Perspective](https://mlsanity.com/paper/2102.10032/) (2021)
- [WaveCNet: Wavelet Integrated CNNs to Suppress Aliasing Effect for Noise-Robust Image Classification](https://mlsanity.com/paper/2107.13335/) (2021)
- [Geometric Scattering on Measure Spaces](https://mlsanity.com/paper/2208.08561/) (2022)
- [A Principled Design of Image Representation: Towards Forensic Tasks](https://mlsanity.com/paper/2203.00913/) (2022)
- [Graphon Pooling for Reducing Dimensionality of Signals and Convolutional Operators on Graphs](https://mlsanity.com/paper/2212.08171/) (2022)
- [Canonical Factors for Hybrid Neural Fields](https://mlsanity.com/paper/2308.15461/) (2023)
- [Stability of Aggregation Graph Neural Networks](https://mlsanity.com/paper/2207.03678/) (2022)
- [Riesz feature representation: scale equivariant scattering network for classification tasks](https://mlsanity.com/paper/2307.08467/) (2023)
- [Monogenic Wavelet Scattering Network for Texture Image Classification](https://mlsanity.com/paper/2202.12491/) (2022)
- [Stability of the scattering transform for deformations with minimal regularity](https://mlsanity.com/paper/2205.11142/) (2022)
- [Enhancing the accuracies by performing pooling decisions adjacent to the output layer](https://mlsanity.com/paper/2303.05800/) (2023)
- [WaveScat: Wavelet Scattering Front-Ends with Self-Supervised Features for Speech Deepfake Detection](https://mlsanity.com/paper/2602.02980/) (2026)
- [Error Bounds for a Matrix-Vector Product Approximation with Deep ReLU Neural Networks](https://mlsanity.com/paper/2111.12963/) (2021)
- [Resource-Efficient Invariant Networks: Exponential Gains by Unrolled Optimization](https://mlsanity.com/paper/2203.05006/) (2022)
- [Dimension-adaptive machine-learning-based quantum state reconstruction](https://mlsanity.com/paper/2205.05804/) (2022)
- [On the Shift Invariance of Max Pooling Feature Maps in Convolutional Neural Networks](https://mlsanity.com/paper/2209.11740/) (2022)
- [Limitless stability for Graph Convolutional Networks](https://mlsanity.com/paper/2301.11443/) (2023)
- [Graph Scattering beyond Wavelet Shackles](https://mlsanity.com/paper/2301.11456/) (2023)
- [Manifold Filter-Combine Networks](https://mlsanity.com/paper/2307.04056/) (2023)
- [Separation Capacity of Scattering Networks](https://mlsanity.com/paper/2606.30822/) (2026)
- [Generalized moduli of continuity under irregular or random deformations via multiscale analysis](https://mlsanity.com/paper/2104.11977/) (2021)
- [Mitigating severe over-parameterization in deep convolutional neural networks through forced feature abstraction and compression with an entropy-based heuristic](https://mlsanity.com/paper/2106.14190/) (2021)
- [The Scattering Transform Network with Generalized Morse Wavelets and Its Application to Music Genre Classification](https://mlsanity.com/paper/2206.07857/) (2022)
- [Computational Modeling of Deep Multiresolution-Fractal Texture and Its Application to Abnormal Brain Tissue Segmentation](https://mlsanity.com/paper/2306.04754/) (2023)
- [Dataset Condensation Driven Machine Unlearning](https://mlsanity.com/paper/2402.00195/) (2024)
- [Hierarchical Invariance for Robust and Interpretable Vision Tasks at Larger Scales](https://mlsanity.com/paper/2402.15430/) (2024)
- [Energy Propagation in Scattering Convolution Networks Can Be Arbitrarily Slow](https://mlsanity.com/paper/2406.05121/) (2024)
- [VDW-GNNs: Vector diffusion wavelets for geometric graph neural networks](https://mlsanity.com/paper/2510.01022/) (2025)

## Abstract (arXiv)

Deep convolutional neural networks have led to breakthrough results in numerous practical machine learning tasks such as classification of images in the ImageNet data set, control-policy-learning to play Atari games or the board game Go, and image captioning. Many of these applications first perform feature extraction and then feed the results thereof into a trainable classifier. The mathematical analysis of deep convolutional neural networks for feature extraction was initiated by Mallat, 2012. Specifically, Mallat considered so-called scattering networks based on a wavelet transform followed by the modulus non-linearity in each network layer, and proved translation invariance (asymptotically in the wavelet scale parameter) and deformation stability of the corresponding feature extractor. This paper complements Mallat's results by developing a theory that encompasses general convolutional transforms, or in more technical parlance, general semi-discrete frames (including Weyl-Heisenberg filters, curvelets, shearlets, ridgelets, wavelets, and learned filters), general Lipschitz-continuous non-linearities (e.g., rectified linear units, shifted logistic sigmoids, hyperbolic tangents, and modulus functions), and general Lipschitz-continuous pooling operators emulating, e.g., sub-sampling and averaging. In addition, all of these elements can be different in different network layers. For the resulting feature extractor we prove a translation invariance result of vertical nature in the sense of the features becoming progressively more translation-invariant with increasing network depth, and we establish deformation sensitivity bounds that apply to signal classes such as, e.g., band-limited functions, cartoon functions, and Lipschitz functions.