mlsanityData to 14 Oct 2026

Approximation Properties of Deep ReLU CNNs

arXiv 2109.00190 ↗First posted 1 Sep 2021cs.LG stat.MLcited by 15

Not on the map. Why?

Abstract (arXiv)

This paper focuses on establishing L2 approximation properties for deep ReLU convolutional neural networks (CNNs) in two-dimensional space. The analysis is based on a decomposition theorem for convolutional kernels with a large spatial size and multi-channels. Given the decomposition result, the property of the ReLU activation function, and a specific structure for channels, a universal approximation theorem of deep ReLU CNNs with classic structure is obtained by showing its connection with one-hidden-layer ReLU neural networks (NNs). Furthermore, approximation properties are obtained for one version of neural networks with ResNet, pre-act ResNet, and MgNet architecture based on connections between these networks.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 23 of its 49 references that are in the map. First posted on arXiv 1 Sep 2021.

Builds on

23 of 49 references.

References · 23

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2016Densely Connected Convolutional Networkscited by 6,720
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  5. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  6. 2016Error bounds for approximations with deep ReLU networkscited by 574
  7. 2014On the Number of Linear Regions of Deep Neural Networkscited by 381
  8. 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 331
  9. 2017The Expressive Power of Neural Networks: A View from the Widthcited by 320
  10. 2016Understanding Deep Neural Networks with Rectified Linear Unitscited by 257
  11. 2016Benefits of depth in neural networkscited by 250
  12. 2018Universality of Deep Convolutional Neural Networkscited by 176
11 more
  1. 2016Why and When Can Deep -- but Not Shallow -- Networks Avoid the Curse of Dimensionality: a Reviewcited by 155
  2. 2019Error bounds for approximations with deep ReLU neural networks in Ws,p normscited by 95
  3. 2016Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with ℓ1 and ℓ0 Controlscited by 84
  4. 2021Characterization of the Variation Spaces Corresponding to Shallow Neural Networkscited by 48
  5. 2019Approximation and Non-parametric Estimation of ResNet-type Convolutional Neural Networkscited by 40
  6. 2018Equivalence of approximation by convolutional neural networks and fully-connected networkscited by 38
  7. 2019Nonlinear Approximation via Compositionscited by 38
  8. 2021Universal Consistency of Deep Convolutional Neural Networkscited by 16
  9. 2020Universal Approximation Theorem for Equivariant Maps by Group CNNscited by 12
  10. 2021ReLU Deep Neural Networks from the Hierarchical Basis Perspectivecited by 12
  11. 2019Constrained Linear Data-feature Mapping for Image Classificationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 2; 2023: 6; 2024: 1; 2025: 3; 2026: 2.

15 papers.

Sort:

  1. 2023MgNO: Efficient Parameterization of Linear Operators via Multigridcited by 40
  2. 2023Theoretical Analysis of Inductive Biases in Deep Convolutional Networkscited by 21
  3. 2024Deeper or Wider: A Perspective from Optimal Generalization Error with Sobolev Losscited by 14
  4. 2023Expressivity and Approximation Properties of Deep Neural Networks with ReLUk Activationcited by 9
  5. 2021An Interpretive Constrained Linear Model for ResNet and MgNetcited by 6
  6. 2022Approximation analysis of CNNs from a feature extraction viewcited by 6
  7. 2025Deep Neural Networks with General Activations: Super-Convergence in Sobolev Normscited by 6
  8. 2023An Enhanced V-cycle MgNet Model for Operator Learning in Numerical Partial Differential Equationscited by 4
  9. 2025Higher Order Approximation Rates for ReLU CNNs in Korobov Spacescited by 4
  10. 2023FV-MgNet: Fully Connected V-cycle MgNet for Interpretable Time Series Forecastingcited by 2
  11. 2025Two-Dimensional Deep ReLU CNN Approximation for Korobov Functions: A Constructive Approachcited by 2
  12. 2022Optimal Convergence Rates of Deep Convolutional Neural Networks: Additive Ridge Functionscited by 1
3 more
  1. 2023Balanced Group Convolution: An Improved Group Convolution Based on Approximability Estimatescited by 0
  2. 2026Progressive Approximation in Deep Residual Networks: Theory and Validationcited by 0
  3. 2026Neural Flow Operators can Approximate any Operator: Abstract Frameworks and Universal Approximationscited by 0