# Approximating smooth functions by deep neural networks with sigmoid activation function

arXiv: https://arxiv.org/abs/2010.04596 · HTML: https://mlsanity.com/paper/2010.04596/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Oct 2020.

## Built on by (most cited first, 18 of 18)

- [Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activations](https://mlsanity.com/paper/2206.09527/) (2022)
- [On the rate of convergence of a classifier based on a Transformer encoder](https://mlsanity.com/paper/2111.14574/) (2021)
- [Estimation of a regression function on a manifold by fully connected deep neural networks](https://mlsanity.com/paper/2107.09532/) (2021)
- [Frequency-adaptive Multi-scale Deep Neural Networks](https://mlsanity.com/paper/2410.00053/) (2024)
- [CovNet: Covariance Networks for Functional Data on Multidimensional Domains](https://mlsanity.com/paper/2104.05021/) (2021)
- [Theoretical guarantees for neural control variates in MCMC](https://mlsanity.com/paper/2304.01111/) (2023)
- [Optimality and Adaptivity of Deep Neural Features for Instrumental Variable Regression](https://mlsanity.com/paper/2501.04898/) (2025)
- [Analysis of Structured Deep Kernel Networks](https://mlsanity.com/paper/2105.07228/) (2021)
- [Fitted value iteration methods for bicausal optimal transport](https://mlsanity.com/paper/2306.12658/) (2023)
- [Deep Learning for Markov Chains: Lyapunov Functions, Poisson's Equation, and Stationary Distributions](https://mlsanity.com/paper/2508.16737/) (2025)
- [On the VC dimension of deep group convolutional neural networks](https://mlsanity.com/paper/2410.15800/) (2024)
- [Second-order Symmetric Non-negative Latent Factor Analysis](https://mlsanity.com/paper/2203.02088/) (2022)
- [Smooth Mathematical Function from Compact Neural Networks](https://mlsanity.com/paper/2301.00181/) (2022)
- [On the growth of the parameters of approximating ReLU neural networks](https://mlsanity.com/paper/2406.14936/) (2024)
- [Statistically guided deep learning](https://mlsanity.com/paper/2504.08489/) (2025)
- [Integral Signatures of Activation Functions: A 9-Dimensional Taxonomy and Stability Theory for Deep Learning](https://mlsanity.com/paper/2510.08456/) (2025)
- [MAED: Mathematical Activation Error Detection for Mitigating Physical Fault Attacks in DNN Inference](https://mlsanity.com/paper/2603.18120/) (2026)
- [On the Regularity and Generalization of One-Step Wasserstein-guided Generative Models for PDE-Induced Measures](https://mlsanity.com/paper/2605.21388/) (2026)

## Abstract (arXiv)

We study the power of deep neural networks (DNNs) with sigmoid activation function. Recently, it was shown that DNNs approximate any d-dimensional, smooth function on a compact set with a rate of order W-p/d, where W is the number of nonzero weights in the network and p is the smoothness of the function. Unfortunately, these rates only hold for a special class of sparsely connected DNNs. We ask ourselves if we can show the same approximation rate for a simpler and more general class, i.e., DNNs which are only defined by its width and depth. In this article we show that DNNs with fixed depth and a width of order Md achieve an approximation rate of M-2p. As a conclusion we quantitatively characterize the approximation power of DNNs in terms of the overall weights W0 in the network and show an approximation rate of W0-p/d. This more general result finally helps us to understand which network topology guarantees a special target accuracy.