mlsanityData to 7 Oct 2026

Nonlinear Weighted Directed Acyclic Graph and A Priori Estimates for Neural Networks

arXiv 2103.16355 ↗First posted 30 Mar 2021cs.LG stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

In an attempt to better understand structural benefits and generalization power of deep neural networks, we firstly present a novel graph theoretical formulation of neural network models, including fully connected, residual network (ResNet) and densely connected networks (DenseNet). Secondly, we extend the error analysis of the population risk for two layer network citeew2019prioriTwo and ResNet citee2019prioriRes to DenseNet, and show further that for neural networks satisfying certain mild conditions, similar estimates can be obtained. These estimates are a priori in nature since they depend sorely on the information prior to the training process, in particular, the bounds for the estimation errors are independent of the input dimension.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 31 references that are in the map. First posted on arXiv 30 Mar 2021.

Builds on

23 of 31 references.

References · 23

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Densely Connected Convolutional Networkscited by 6,691
  3. 2016Deep Networks with Stochastic Depthcited by 973
  4. 2017Spectrally-normalized margin bounds for neural networkscited by 681
  5. 2017Exploring Generalization in Deep Learningcited by 595
  6. 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 562
  7. 2018Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layerscited by 340
  8. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 309
  9. 2017Size-Independent Sample Complexity of Neural Networkscited by 289
  10. 2018Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Datacited by 260
  11. 2016FractalNet: Ultra-Deep Neural Networks without Residualscited by 216
  12. 2015Training Very Deep Networkscited by 172
11 more
  1. 2016Identity Matters in Deep Learningcited by 107
  2. 2017Fisher-Rao Metric, Geometry, and Complexity of Neural Networkscited by 99
  3. 2020Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don'tcited by 94
  4. 2017Implicit Regularization in Deep Learningcited by 90
  5. 2018A Priori Estimates of the Population Risk for Two-layer Neural Networkscited by 76
  6. 2020Representation formulas and pointwise properties for Barron functionscited by 62
  7. 2019Generalization Error Bounds of Gradient Descent for Learning Over-parameterized Deep ReLU Networkscited by 60
  8. 2017SGD Learns the Conjugate Kernel Class of the Networkcited by 54
  9. 2020Kolmogorov Width Decay and Poor Approximators in Machine Learning: Shallow Neural Networks, Random Feature Models and Neural Tangent Kernelscited by 19
  10. 2019A Priori Estimates of the Population Risk for Residual Networkscited by 14
  11. 2020A priori estimates for classification problems using neural networkscited by 5

Built on by

Not cited yet.