mlsanityData to 14 Oct 2026

Smooth function approximation by deep neural networks with general activation functions

arXiv 1906.06903 ↗First posted 17 Jun 2019stat.ML cs.LGcited by 30

Not on the map. Why?

Abstract (arXiv)

There has been a growing interest in expressivity of deep neural networks. However, most of the existing work about this topic focuses only on the specific activation function such as ReLU or sigmoid. In this paper, we investigate the approximation ability of deep neural networks with a broad class of activation functions. This class of activation functions includes most of frequently used activation functions. We derive the required depth, width and sparsity of a deep neural network to approximate any H"older smooth function upto a given approximation error for the large class of activation functions. Based on our approximation error analysis, we derive the minimax optimality of the deep neural network estimators with the general activation functions in both regression and classification problems.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). First posted on arXiv 17 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 5; 2022: 4; 2023: 6; 2024: 5; 2025: 6; 2026: 4.

30 papers.

Sort:

  1. 2021On the approximation of functions by tanh neural networkscited by 81
  2. 2021Besov Function Approximation and Binary Classification on Low-Dimensional Manifolds Using Convolutional Residual Networkscited by 32
  3. 2023On Deep Generative Models for Approximation and Estimation of Distributions on Manifoldscited by 13
  4. 2021A likelihood approach to nonparametric estimation of a singular distribution using deep generative modelscited by 12
  5. 2023Excess risk bound for deep learning under weak dependencecited by 9
  6. 2022Approximation speed of quantized vs. unquantized ReLU neural networks and beyondcited by 8
  7. 2023Sparse-penalized deep neural networks estimator under weak dependencecited by 8
  8. 2024Learning smooth functions in high dimensions: from sparse polynomials to deep neural networkscited by 7
  9. 2024Robust deep learning from weakly dependent datacited by 5
  10. 2025Orthogonal Representation Learning for Estimating Causal Quantitiescited by 5
  11. 2021CovNet: Covariance Networks for Functional Data on Multidimensional Domainscited by 4
  12. 2022Neural Networks with A La Carte Selection of Activation Functionscited by 4
18 more
  1. 2023Intrinsic and extrinsic deep learning on manifoldscited by 4
  2. 2024A Likelihood Based Approach to Distribution Regression Using Conditional Deep Generative Modelscited by 4
  3. 2024Minimax optimality of deep neural networks on dependent data via PAC-Bayes boundscited by 4
  4. 2025Deep Generative Models: Complexity, Dimensionality, and Approximationcited by 4
  5. 2022On the Suitability of Neural Networks as Building Blocks for The Design of Efficient Learned Indexescited by 3
  6. 2024Deep learning from strongly mixing observations: Sparse-penalized regularization and minimax optimalitycited by 3
  7. 2021Analysis of Structured Deep Kernel Networkscited by 2
  8. 2022What Has Been Enhanced in my Knowledge-Enhanced Language Model?cited by 2
  9. 2023Deep learning for psi-weakly dependent processescited by 2
  10. 2023Penalized deep neural networks estimator with general loss functions under weak dependencecited by 2
  11. 2025Learned Bayesian Cram'er-Rao Bound for Unknown Measurement Models Using Score Neural Networkscited by 2
  12. 2025A general framework for deep learningcited by 2
  13. 2025ReLU integral probability metric and its applicationscited by 1
  14. 2025Integral Signatures of Activation Functions: A 9-Dimensional Taxonomy and Stability Theory for Deep Learningcited by 0
  15. 2026Deep regression learning from dependent observations with minimum error entropy principlecited by 0
  16. 2026A Composite Activation Function for Learning Stable Binary Representationscited by 0
  17. 2026Modeling Covariate Transition for Efficient Estimation of Longitudinal Treatment Effects in Randomized Experimentscited by 0
  18. 2026Adaptive deep nonparametric regression from dependent data under covariate shiftcited by 0