# Construction of neural networks for realization of localized deep learning

arXiv: https://arxiv.org/abs/1803.03503 · HTML: https://mlsanity.com/paper/1803.03503/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Mar 2018.

## Built on by (most cited first, 7 of 7)

- [Deep Network Approximation: Beyond ReLU to Diverse Activation Functions](https://mlsanity.com/paper/2307.06555/) (2023)
- [Neural Network Architecture Beyond Width and Depth](https://mlsanity.com/paper/2205.09459/) (2022)
- [Deep Network Approximation in Terms of Intrinsic Parameters](https://mlsanity.com/paper/2111.07964/) (2021)
- [On Enhancing Expressive Power via Compositions of Single Fixed-Size ReLU Network](https://mlsanity.com/paper/2301.12353/) (2023)
- [Fourier Multi-Component and Multi-Layer Neural Networks: Unlocking High-Frequency Potential](https://mlsanity.com/paper/2502.18959/) (2025)
- [Structured and Balanced Multi-Component and Multi-Layer Neural Networks](https://mlsanity.com/paper/2407.00765/) (2024)
- [Sampling Complexity of Deep Approximation Spaces](https://mlsanity.com/paper/2312.13379/) (2023)

## Abstract (arXiv)

The subject of deep learning has recently attracted users of machine learning from various disciplines, including: medical diagnosis and bioinformatics, financial market analysis and online advertisement, speech and handwriting recognition, computer vision and natural language processing, time series forecasting, and search engines. However, theoretical development of deep learning is still at its infancy. The objective of this paper is to introduce a deep neural network (also called deep-net) approach to localized manifold learning, with each hidden layer endowed with a specific learning task. For the purpose of illustrations, we only focus on deep-nets with three hidden layers, with the first layer for dimensionality reduction, the second layer for bias reduction, and the third layer for variance reduction. A feedback component also designed to eliminate outliers. The main theoretical result in this paper is the order mathcal Oleft(m-2s/(2s+d)right) of approximation of the regression function with regularity s, in terms of the number m of sample points, where the (unknown) manifold dimension d replaces the dimension D of the sampling (Euclidean) space for shallow nets.