# Approximation analysis of CNNs from a feature extraction view

arXiv: https://arxiv.org/abs/2210.09041 · HTML: https://mlsanity.com/paper/2210.09041/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 30 references that are in the map.
- First posted on arXiv 14 Oct 2022.

## Builds on (most cited first, 10 of 10)

- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [Universality of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1805.10769/) (2018)
- [Deep Network Approximation Characterized by Number of Neurons](https://mlsanity.com/paper/1906.05497/) (2019)
- [Optimal Approximation with Sparsely Connected Deep Neural Networks](https://mlsanity.com/paper/1705.01714/) (2017)
- [Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with  ℓ1  and  ℓ0  Controls](https://mlsanity.com/paper/1607.07819/) (2016)
- [Approximation and Non-parametric Estimation of ResNet-type Convolutional Neural Networks](https://mlsanity.com/paper/1903.10047/) (2019)
- [Besov Function Approximation and Binary Classification on Low-Dimensional Manifolds Using Convolutional Residual Networks](https://mlsanity.com/paper/2109.02832/) (2021)
- [Theory of Deep Convolutional Neural Networks II: Spherical Analysis](https://mlsanity.com/paper/2007.14285/) (2020)
- [Theory of Deep Convolutional Neural Networks III: Approximating Radial Functions](https://mlsanity.com/paper/2107.00896/) (2021)
- [Approximation Properties of Deep ReLU CNNs](https://mlsanity.com/paper/2109.00190/) (2021)

## Built on by (most cited first, 6 of 6)

- [Theoretical Analysis of Inductive Biases in Deep Convolutional Networks](https://mlsanity.com/paper/2305.08404/) (2023)
- [Higher Order Approximation Rates for ReLU CNNs in Korobov Spaces](https://mlsanity.com/paper/2501.11275/) (2025)
- [Convergence Analysis for Deep Sparse Coding via Convolutional Neural Networks](https://mlsanity.com/paper/2408.05540/) (2024)
- [CHEM: Estimating and Understanding Hallucinations in Deep Learning for Image Processing](https://mlsanity.com/paper/2512.09806/) (2025)
- [Sparse-Aware Neural Networks for Nonlinear Functionals: Mitigating the Exponential Dependence on Dimension](https://mlsanity.com/paper/2604.06774/) (2026)
- [Neural Flow Operators can Approximate any Operator: Abstract Frameworks and Universal Approximations](https://mlsanity.com/paper/2605.22557/) (2026)

## Abstract (arXiv)

Deep learning based on deep neural networks has been very successful in many practical applications, but it lacks enough theoretical understanding due to the network architectures and structures. In this paper we establish some analysis for linear feature extraction by a deep multi-channel convolutional neural networks (CNNs), which demonstrates the power of deep learning over traditional linear transformations, like Fourier, wavelets, redundant dictionary coding methods. Moreover, we give an exact construction presenting how linear features extraction can be conducted efficiently with multi-channel CNNs. It can be applied to lower the essential dimension for approximating a high dimensional function. Rates of function approximation by such deep networks implemented with channels and followed by fully-connected layers are investigated as well. Harmonic analysis for factorizing linear features into multi-resolution convolutions plays an essential role in our work. Nevertheless, a dedicate vectorization of matrices is constructed, which bridges 1D CNN and 2D CNN and allows us to have corresponding 2D analysis.