# Universal Consistency of Deep Convolutional Neural Networks

arXiv: https://arxiv.org/abs/2106.12498 · HTML: https://mlsanity.com/paper/2106.12498/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 28 references that are in the map.
- First posted on arXiv 23 Jun 2021.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Convergence Theory for Deep Learning via Over-Parameterization](https://mlsanity.com/paper/1811.03962/) (2018)
- [A Mean Field View of the Landscape of Two-Layers Neural Networks](https://mlsanity.com/paper/1804.06561/) (2018)
- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks](https://mlsanity.com/paper/1703.02930/) (2017)
- [Universality of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1805.10769/) (2018)
- [Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activations](https://mlsanity.com/paper/1708.02691/) (2017)
- [ECG Heartbeat Classification: A Deep Transferable Representation](https://mlsanity.com/paper/1805.00794/) (2018)
- [Approximation and Non-parametric Estimation of ResNet-type Convolutional Neural Networks](https://mlsanity.com/paper/1903.10047/) (2019)
- [Generalization and Expressivity for Deep Nets](https://mlsanity.com/paper/1803.03772/) (2018)
- [Realization of spatial sparseness by deep ReLU nets with massive data](https://mlsanity.com/paper/1912.07464/) (2019)
- [Realizing data features by deep nets](https://mlsanity.com/paper/1901.00130/) (2019)

## Built on by (most cited first, 16 of 16)

- [Stability and Generalization Analysis of Gradient Methods for Shallow Neural Networks](https://mlsanity.com/paper/2209.09298/) (2022)
- [Approximation Properties of Deep ReLU CNNs](https://mlsanity.com/paper/2109.00190/) (2021)
- [Optimal rates of approximation by shallow ReLUk neural networks and applications to nonparametric regression](https://mlsanity.com/paper/2304.01561/) (2023)
- [On the rates of convergence for learning with convolutional neural networks](https://mlsanity.com/paper/2403.16459/) (2024)
- [Learning Ability of Interpolating Deep Convolutional Neural Networks](https://mlsanity.com/paper/2210.14184/) (2022)
- [Deep Convolutional Neural Networks with Zero-Padding: Feature Extraction and Learning](https://mlsanity.com/paper/2307.16203/) (2023)
- [Generalization Analysis of Online Stochastic Gradient Descent for Overparameterized Two-Layer Neural Networks](https://mlsanity.com/paper/2407.07670/) (2024)
- [Higher Order Approximation Rates for ReLU CNNs in Korobov Spaces](https://mlsanity.com/paper/2501.11275/) (2025)
- [Deep Learning is Provably Robust to Symmetric Label Noise](https://mlsanity.com/paper/2210.15083/) (2022)
- [Optimal Convergence Rates of Deep Convolutional Neural Networks: Additive Ridge Functions](https://mlsanity.com/paper/2202.12119/) (2022)
- [Ensemble Multi-Quantiles: Adaptively Flexible Distribution Prediction for Uncertainty Quantification](https://mlsanity.com/paper/2211.14545/) (2022)
- [On the Connection between Lp and Risk Consistency and its Implications on Regularized Kernel Methods](https://mlsanity.com/paper/2303.15210/) (2023)
- [Optimal Approximation and Learning Rates for Deep Convolutional Neural Networks](https://mlsanity.com/paper/2308.03259/) (2023)
- [Component-based Sketching for Deep ReLU Nets](https://mlsanity.com/paper/2409.14174/) (2024)
- [Transformers Are Universally Consistent](https://mlsanity.com/paper/2505.24531/) (2025)
- [Ratio-based Loss Functions](https://mlsanity.com/paper/2605.05808/) (2026)

## Abstract (arXiv)

Compared with avid research activities of deep convolutional neural networks (DCNNs) in practice, the study of theoretical behaviors of DCNNs lags heavily behind. In particular, the universal consistency of DCNNs remains open. In this paper, we prove that implementing empirical risk minimization on DCNNs with expansive convolution (with zero-padding) is strongly universally consistent. Motivated by the universal consistency, we conduct a series of experiments to show that without any fully connected layers, DCNNs with expansive convolution perform not worse than the widely used deep neural networks with hybrid structure containing contracting (without zero-padding) convolution layers and several fully connected layers.