mlsanityData to 7 Oct 2026

Approximation bounds for norm constrained neural networks with applications to regression and GANs

arXiv 2201.09418 ↗First posted 24 Jan 2022cs.LG cs.NA math.NAcited by 21

Not on the map. Why?

Abstract (arXiv)

This paper studies the approximation capacity of ReLU neural networks with norm constraint on the weights. We prove upper and lower bounds on the approximation error of these networks for smooth function classes. The lower bound is derived through the Rademacher complexity of neural networks, which may be of independent interest. We apply these approximation bounds to analyze the convergences of regression using norm constrained neural networks and distribution estimation by GANs. In particular, we obtain convergence rates for over-parameterized neural networks. It is also shown that GANs can achieve optimal rate of learning probability distributions, when the discriminator is a properly chosen norm constrained neural network.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 35 of its 63 references that are in the map. First posted on arXiv 24 Jan 2022.

Builds on

35 of 63 references.

References · 35

  1. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  2. 2017Improved Training of Wasserstein GANscited by 2,223
  3. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  4. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 784
  5. 2017Spectrally-normalized margin bounds for neural networkscited by 681
  6. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 572
  7. 2016Error bounds for approximations with deep ReLU networkscited by 567
  8. 2017Towards Principled Methods for Training Generative Adversarial Networkscited by 460
  9. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 431
  10. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 309
  11. 2017Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networkscited by 304
  12. 2015Norm-Based Capacity Control in Neural Networkscited by 299
23 more
  1. 2017Size-Independent Sample Complexity of Neural Networkscited by 289
  2. 2017Parseval Networks: Improving Robustness to Adversarial Examplescited by 288
  3. 2020Loss landscapes and optimization in over-parameterized non-linear systems and neural networkscited by 231
  4. 2017Optimal approximation of piecewise smooth functions using deep ReLU neural networkscited by 176
  5. 2018Optimal approximation of continuous functions by very deep ReLU networkscited by 165
  6. 2020Deep Network Approximation for Smooth Functionscited by 163
  7. 2019Deep Network Approximation Characterized by Number of Neuronscited by 120
  8. 2015Path-SGD: Path-Normalized Optimization in Deep Neural Networkscited by 117
  9. 2017Optimal Approximation with Sparsely Connected Deep Neural Networkscited by 90
  10. 2020The Kolmogorov-Arnold representation theorem revisitedcited by 89
  11. 2019The phase diagram of approximation rates for deep neural networkscited by 86
  12. 2018How Well Generative Adversarial Networks Learn Distributionscited by 78
  13. 2017On the regularization of Wasserstein GANscited by 65
  14. 2018Limitations of the Lipschitz constant as a defense against adversarial examplescited by 48
  15. 2021On the Proof of Global Convergence of Gradient Descent for Deep ReLU Networks with Linear Widthscited by 44
  16. 2021An error analysis of generative adversarial networks for learning distributionscited by 35
  17. 2017Sobolev GANcited by 29
  18. 2018Nonparametric Density Estimation under Adversarial Lossescited by 26
  19. 2021On the capacity of deep generative networks for approximating distributionscited by 19
  20. 2019Complexity, Statistical Risk, and Metric Entropy of Deep Nets Using Total Path Variationcited by 14
  21. 2020Constructive Universal High-Dimensional Distribution Generation through Deep ReLU Networkscited by 13
  22. 2020Approximation in shift-invariant spaces with deep ReLU neural networkscited by 12
  23. 2021High-Dimensional Distribution Generation Through Deep Neural Networkscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 4; 2024: 6; 2025: 3; 2026: 6.

21 papers.

Sort:

By area:

  1. 2023Nonparametric regression using over-parameterized shallow ReLU neural networkscited by 24
  2. 2023Optimal rates of approximation by shallow ReLUk neural networks and applications to nonparametric regressioncited by 15
  3. 2024On the optimal approximation of Sobolev and Besov functions using deep ReLU neural networkscited by 13
  4. 2024On the rates of convergence for learning with convolutional neural networkscited by 10
  5. 2024Deep Transfer Learning: Model Framework and Error Analysiscited by 4
  6. 2025Wasserstein Distributionally Robust Nonparametric Regressioncited by 3
  7. 2025Some Super-approximation Rates of ReLU Neural Networks for Korobov Functionscited by 3
  8. 2026Approximation and learning of anisotropic and mixed smooth functions by deep ReLU neural networkscited by 3
  9. 2023Non-Asymptotic Bounds for Adversarial Excess Risk under Misspecified Modelscited by 2
  10. 2026Operator Learning for Families of Finite-State Mean-Field Gamescited by 2
  11. 2026Learning Sparse Compositional Functions with Norm-Constrained Neural Networkscited by 2
  12. 2022Learning Distributions by Generative Adversarial Networks: Approximation and Generalizationcited by 1
9 more
  1. 2023Neural Network Approximation for Pessimistic Offline Reinforcement Learningcited by 1
  2. 2024Theoretical Insights into CycleGAN: Analyzing Approximation and Estimation Errors in Unpaired Data Generationcited by 1
  3. 2024Adv-SSL: Adversarial Self-Supervised Representation Learning with Theoretical Guaranteescited by 1
  4. 2025Depth-induced NTK: Bridging Over-parameterized Neural Networks and Deep Neural Kernelscited by 1
  5. 2022Distribution estimation and change-point estimation for time series via DNN-based GANscited by 0
  6. 2024Consistency for Large Neural Networks: Regression and Classificationcited by 0
  7. 2026Approximation of Analytic Functions by ReLU Neural Networks with Adjustable Depth and Widthcited by 0
  8. 2026Error bounds in Sobolev norms for approximations with norm constrained ReLU neural networkscited by 0
  9. 2026Neural Approximation by Function Composition: Rigidity and Doubly Exponential Convergencecited by 0