mlsanityData to 14 Oct 2026

Nearly Optimal VC-Dimension and Pseudo-Dimension Bounds for Deep Neural Network Derivatives

arXiv 2305.08466 ↗First posted 15 May 2023cs.LG cs.NA math.NAcited by 30

Not on the map. Why?

Abstract (arXiv)

This paper addresses the problem of nearly optimal Vapnik--Chervonenkis dimension (VC-dimension) and pseudo-dimension estimations of the derivative functions of deep neural networks (DNNs). Two important applications of these estimations include: 1) Establishing a nearly tight approximation result of DNNs in the Sobolev space; 2) Characterizing the generalization error of machine learning methods with loss functions involving function derivatives. This theoretical investigation fills the gap of learning error estimations for a wide range of physics-informed machine learning models and applications including generative models, solving partial differential equations, operator learning, network compression, distillation, regularization, etc.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Builds on 18 of its 43 references that are in the map. First posted on arXiv 15 May 2023.

Builds on

18 of 43 references.

References · 18

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
  3. 2015Policy Distillationcited by 440
  4. 2017Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networkscited by 311
  5. 2017Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equationscited by 191
  6. 2020Deep Network Approximation for Smooth Functionscited by 164
  7. 2014Towards Deep Neural Network Architectures Robust to Adversarial Examplescited by 146
  8. 2017Sobolev Training for Neural Networkscited by 143
  9. 2019Deep Network Approximation Characterized by Number of Neuronscited by 122
  10. 2021Optimal Approximation Rate of ReLU Networks in terms of Width and Depthcited by 96
  11. 2019Error bounds for approximations with deep ReLU neural networks in Ws,p normscited by 95
  12. 2021On the approximation of functions by tanh neural networkscited by 81
6 more
  1. 2018Banach Wasserstein GANcited by 56
  2. 2022Deep Nonparametric Estimation of Operators between Infinite Dimensional Spacescited by 45
  3. 2019Nonlinear Approximation via Compositionscited by 38
  4. 2016Deep Model Compression: Distilling Knowledge from Noisy Teacherscited by 36
  5. 2017Sobolev GANcited by 29
  6. 2021Error Estimates for the Deep Ritz Method with Boundary Penaltycited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 9; 2025: 10; 2026: 7.

30 papers.

Sort:

By area:

  1. 2024Convergence of Continuous Normalizing Flows for Learning Probability Distributionscited by 26
  2. 2024Deeper or Wider: A Perspective from Optimal Generalization Error with Sobolev Losscited by 14
  3. 2023On the Optimal Expressive Power of ReLU DNNs and Its Application in Approximation with Kolmogorov Superposition Theoremcited by 13
  4. 2024Automatic Differentiation is Essential in Training Neural Networks for Solving Differential Equationscited by 9
  5. 2023Deep Neural Networks and Finite Elements of Any Order on Arbitrary Dimensionscited by 8
  6. 2025Deep Neural Networks with General Activations: Super-Convergence in Sobolev Normscited by 6
  7. 2023Homotopy Relaxation Training Algorithms for Infinite-Width Two-Layer ReLU Neural Networkscited by 4
  8. 2023Optimal Deep Neural Network Approximation for Korobov Functions with respect to Sobolev Normscited by 4
  9. 2024Quantifying Training Difficulty and Accelerating Convergence in Neural Network-Based PDE Solverscited by 4
  10. 2025An Imbalanced Learning-based Sampling Method for Physics-informed Neural Networkscited by 4
  11. 2025RL-Selector: Reinforcement Learning-Guided Data Selection via Redundancy Assessmentcited by 3
  12. 2025Some Super-approximation Rates of ReLU Neural Networks for Korobov Functionscited by 3
18 more
  1. 2026Learning Sparse Compositional Functions with Norm-Constrained Neural Networkscited by 3
  2. 2024Newton Informed Neural Operator for Computing Multiple Solutions of Nonlinear Partials Differential Equationscited by 2
  3. 2025Expressive Power of Deep Networks on Manifolds: Simultaneous Approximationcited by 2
  4. 2024Error Analysis of Three-Layer Neural Network Trained with PGD for Deep Ritz Methodcited by 1
  5. 2024Blessing of Dimensionality for Approximating Sobolev Classes on Manifoldscited by 1
  6. 2024DeepONet for Solving Nonlinear Partial Differential Equations with Physics-Informed Trainingcited by 1
  7. 2025Curse of Dimensionality in Neural Network Optimizationcited by 1
  8. 2025Statistical Learning Guarantees for Group-Invariant Barron Functionscited by 1
  9. 2026A Quantitative Approximation Framework for Flow Distillation in Diffusion Modelscited by 1
  10. 2024Generalization bounds for regression and classification on adaptive covering input domainscited by 0
  11. 2025Generalizability of Neural Networks Minimizing Empirical Risk Based on Expressive Abilitycited by 0
  12. 2025Energy Approach from varepsilon-Graph to Continuum Diffusion Model with Connectivity Functionalcited by 0
  13. 2025On the Dimension-Free Approximation of Deep Neural Networks for Symmetric Korobov Functionscited by 0
  14. 2026Understanding Generalization through Decision Pattern Shiftcited by 0
  15. 2026A Rigorous, Tractable Measure of Model Complexitycited by 0
  16. 2026Neural Flow Operators can Approximate any Operator: Abstract Frameworks and Universal Approximationscited by 0
  17. 2026Approximation of Analytic Functions by ReLU Neural Networks with Adjustable Depth and Widthcited by 0
  18. 2026Error bounds in Sobolev norms for approximations with norm constrained ReLU neural networkscited by 0