# Deep Neural Networks for Rotation-Invariance Approximation and Learning

arXiv: https://arxiv.org/abs/1904.01814 · HTML: https://mlsanity.com/paper/1904.01814/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Apr 2019.

## Built on by (most cited first, 10 of 10)

- [Theory of Deep Convolutional Neural Networks III: Approximating Radial Functions](https://mlsanity.com/paper/2107.00896/) (2021)
- [Expressivity and Approximation Properties of Deep Neural Networks with ReLUk Activation](https://mlsanity.com/paper/2312.16483/) (2023)
- [Nonlinear functional regression by functional deep neural network with kernel embedding](https://mlsanity.com/paper/2401.02890/) (2024)
- [Generalization Performance of Empirical Risk Minimization on Over-parameterized Deep ReLU Nets](https://mlsanity.com/paper/2111.14039/) (2021)
- [SignReLU neural network and its approximation ability](https://mlsanity.com/paper/2210.10264/) (2022)
- [Deep Convolutional Neural Networks with Zero-Padding: Feature Extraction and Learning](https://mlsanity.com/paper/2307.16203/) (2023)
- [Learning Theory of Distribution Regression with Neural Networks](https://mlsanity.com/paper/2307.03487/) (2023)
- [Sampling Complexity of Deep Approximation Spaces](https://mlsanity.com/paper/2312.13379/) (2023)
- [Component-based Sketching for Deep ReLU Nets](https://mlsanity.com/paper/2409.14174/) (2024)
- [Lifting the Veil: Unlocking the Power of Depth in Q-learning](https://mlsanity.com/paper/2310.17915/) (2023)

## Abstract (arXiv)

Based on the tree architecture, the objective of this paper is to design deep neural networks with two or more hidden layers (called deep nets) for realization of radial functions so as to enable rotational invariance for near-optimal function approximation in an arbitrarily high dimensional Euclidian space. It is shown that deep nets have much better performance than shallow nets (with only one hidden layer) in terms of approximation accuracy and learning capabilities. In particular, for learning radial functions, it is shown that near-optimal rate can be achieved by deep nets but not by shallow nets. Our results illustrate the necessity of depth in neural network design for realization of rotation-invariance target functions.