# Equivalence of approximation by convolutional neural networks and fully-connected networks

arXiv: https://arxiv.org/abs/1809.00973 · HTML: https://mlsanity.com/paper/1809.00973/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Sep 2018.

## Built on by (most cited first, 30 of 38)

- [Convolutional Neural Operators for robust and accurate learning of PDEs](https://mlsanity.com/paper/2302.01178/) (2023)
- [Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practice](https://mlsanity.com/paper/2203.05962/) (2022)
- [Besov Function Approximation and Binary Classification on Low-Dimensional Manifolds Using Convolutional Residual Networks](https://mlsanity.com/paper/2109.02832/) (2021)
- [Universal Approximation Under Constraints is Possible with Transformers](https://mlsanity.com/paper/2110.03303/) (2021)
- [Theory of Deep Convolutional Neural Networks III: Approximating Radial Functions](https://mlsanity.com/paper/2107.00896/) (2021)
- [Approximation Properties of Deep ReLU CNNs](https://mlsanity.com/paper/2109.00190/) (2021)
- [A Brief Survey on the Approximation Theory for Sequence Modelling](https://mlsanity.com/paper/2302.13752/) (2023)
- [Optimal rates of approximation by shallow ReLUk neural networks and applications to nonparametric regression](https://mlsanity.com/paper/2304.01561/) (2023)
- [Fully-Connected Network on Noncompact Symmetric Space and Ridgelet Transform based on Helgason-Fourier Analysis](https://mlsanity.com/paper/2203.01631/) (2022)
- [Convergence of Deep Convolutional Neural Networks](https://mlsanity.com/paper/2109.13542/) (2021)
- [Generalization Error Guaranteed Auto-Encoder-Based Nonlinear Model Reduction for Operator Learning](https://mlsanity.com/paper/2401.10490/) (2024)
- [Universality of Group Convolutional Neural Networks Based on Ridgelet Analysis on Groups](https://mlsanity.com/paper/2205.14819/) (2022)
- [Deep Neural Network Approximation of Invariant Functions through Dynamical Systems](https://mlsanity.com/paper/2208.08707/) (2022)
- [Learning Ability of Interpolating Deep Convolutional Neural Networks](https://mlsanity.com/paper/2210.14184/) (2022)
- [A practical existence theorem for reduced order models based on convolutional autoencoders](https://mlsanity.com/paper/2402.00435/) (2024)
- [On the Universal Approximation Property of Deep Fully Convolutional Neural Networks](https://mlsanity.com/paper/2211.14047/) (2022)
- [Identifiable Equivariant Networks are Layerwise Equivariant](https://mlsanity.com/paper/2601.21645/) (2026)
- [Benefits of Overparameterized Convolutional Residual Networks: Function Approximation under Smoothness Constraint](https://mlsanity.com/paper/2206.04569/) (2022)
- [Tailor: Altering Skip Connections for Resource-Efficient Inference](https://mlsanity.com/paper/2301.07247/) (2023)
- [Universal Regular Conditional Distributions](https://mlsanity.com/paper/2105.07743/) (2021)
- [Analysis of convolutional neural network image classifiers in a rotationally symmetric model](https://mlsanity.com/paper/2205.05500/) (2022)
- [Deep Convolutional Neural Networks with Zero-Padding: Feature Extraction and Learning](https://mlsanity.com/paper/2307.16203/) (2023)
- [Higher Order Approximation Rates for ReLU CNNs in Korobov Spaces](https://mlsanity.com/paper/2501.11275/) (2025)
- [New universal operator approximation theorem for encoder-decoder architectures](https://mlsanity.com/paper/2503.24092/) (2025)
- [On Universality Classes of Equivariant Networks](https://mlsanity.com/paper/2506.02293/) (2025)
- [Universal approximation with complex-valued deep narrow neural networks](https://mlsanity.com/paper/2305.16910/) (2023)
- [Spectral GNN via Two-dimensional (2-D) Graph Convolution](https://mlsanity.com/paper/2404.04559/) (2024)
- [Deep Neural Networks are Adaptive to Function Regularity and Data Distribution in Approximation and Estimation](https://mlsanity.com/paper/2406.05320/) (2024)
- [Equivariant and Steerable Neural Networks: A review with special emphasis on the symmetric group](https://mlsanity.com/paper/2301.03019/) (2023)
- [Approximation Rates and VC-Dimension Bounds for (P)ReLU MLP Mixture of Experts](https://mlsanity.com/paper/2402.03460/) (2024)

## Abstract (arXiv)

Convolutional neural networks are the most widely used type of neural networks in applications. In mathematical analysis, however, mostly fully-connected networks are studied. In this paper, we establish a connection between both network architectures. Using this connection, we show that all upper and lower bounds concerning approximation rates of fully-connected neural networks for functions f ∈ C -- for an arbitrary function class C -- translate to essentially the same bounds concerning approximation rates of convolutional neural networks for functions f ∈ Cequi, with the class Cequi consisting of all translation equivariant functions whose first coordinate belongs to C. All presented results consider exclusively the case of convolutional neural networks without any pooling operation and with circular convolutions, i.e., not based on zero-padding.