mlsanityData to 7 Oct 2026

Convergence analysis of controlled particle systems arising in deep learning: from finite to infinite sample size

arXiv 2404.05185 ↗First posted 8 Apr 2024math.OC cs.LG math.PRcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper deals with a class of neural SDEs and studies the limiting behavior of the associated sampled optimal control problems as the sample size grows to infinity. The neural SDEs with N samples can be linked to the N-particle systems with centralized control. We analyze the Hamilton--Jacobi--Bellman equation corresponding to the N-particle system and establish regularity results which are uniform in N. The uniform regularity estimates are obtained by the stochastic maximum principle and the analysis of a backward stochastic Riccati equation. Using these uniform regularity results, we show the convergence of the minima of the objective functionals and optimal parameters of the neural SDEs as the sample size N tends to infinity. The limiting objects can be identified with suitable functions defined on the Wasserstein space of Borel probability measures. Furthermore, quantitative convergence rates are also obtained.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 43 of its 78 references that are in the map. First posted on arXiv 8 Apr 2024.

Builds on

43 of 78 references.

References · 43

  1. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  2. 2018Neural Ordinary Differential Equationscited by 3,733
  3. 2018A Mean Field View of the Landscape of Two-Layers Neural Networkscited by 463
  4. 2018On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transportcited by 422
  5. 2017Stable Architectures for Deep Neural Networkscited by 340
  6. 2017Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equationscited by 188
  7. 2019A Machine Learning Framework for Solving High-Dimensional Mean Field Game and Mean Field Control Problemscited by 95
  8. 2021Optimal Approximation Rate of ReLU Networks in terms of Width and Depthcited by 95
  9. 2019Deep Learning via Dynamical Systems: An Approximation Perspectivecited by 78
  10. 2017Maximum Principle Based Algorithms for Deep Learningcited by 77
  11. 2018A Priori Estimates of the Population Risk for Two-layer Neural Networkscited by 76
  12. 2018A Mean-Field Optimal Control Formulation of Deep Learningcited by 67
31 more
  1. 2020Neural Network Approximation: Three Hidden Layers Are Enoughcited by 63
  2. 2016Deep Learning Approximation for Stochastic Control Problemscited by 51
  3. 2019On the Convergence of Model Free Learning in Mean Field Gamescited by 45
  4. 2019Deep neural networks, generic universal interpolation, and controlled ODEscited by 28
  5. 2022Uniform-in-time propagation of chaos for mean field Langevin dynamicscited by 23
  6. 2018Deep neural networks algorithms for stochastic control problems on finite horizon: convergence analysiscited by 22
  7. 2021Deep Learning for Mean Field Games and Mean Field Control with Applications to Financecited by 20
  8. 2019Mean-Field Neural ODEs via Relaxed Optimal Controlcited by 16
  9. 2021Scaling Properties of Deep Residual Networkscited by 16
  10. 2018Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applicationscited by 15
  11. 2016Dynamic programming for optimal control of stochastic McKean-Vlasov dynamicscited by 11
  12. 2016Limit theory for controlled McKean-Vlasov dynamicscited by 11
  13. 2021Mean Field Games Master Equations with Non-separable Hamiltonians and Displacement Monotonicitycited by 4
  14. 2021Rate of convergence for particle approximation of PDEs in Wasserstein spacecited by 3
  15. 2019Large Sample Mean-Field Stochastic Optimizationcited by 2
  16. 2020Extended mean field control problem: a propagation of chaos resultcited by 2
  17. 2023On the Optimal Rate for the Convergence Problem in Mean Field Controlcited by 2
  18. 2016EM Algorithm and Stochastic Control in Economicscited by 1
  19. 2018Viscosity Solutions to Parabolic Master Equations and McKean-Vlasov SDEs with Closed-loop Controlscited by 1
  20. 2019Stochastic Control on Space of Random Variablescited by 1
  21. 2021Centralized systemic risk control in the interbank system: Weak formulation and Gamma-convergencecited by 1
  22. 2022An algebraic convergence rate for the optimal control of McKean-Vlasov dynamicscited by 1
  23. 2022Mean Field Games of Controls: Propagation of Monotonicitiescited by 1
  24. 2022Linear-Quadratic Mean Field Games of Controls with Non-Monotone Datacited by 1
  25. 2023Deep Learning for Mean Field Optimal Transportcited by 1
  26. 2023Quantitative convergence for displacement monotone mean field games with controlled volatilitycited by 1
  27. 2023Global Well-Posedness of Displacement Monotone Degenerate Mean Field Games Master Equationscited by 1
  28. 2023Linear-Quadratic Mean Field Control with Non-Convex Datacited by 1
  29. 2023Sharp convergence rates for mean field control in the region of strong regularitycited by 1
  30. 2024On classical solutions and canonical transformations for Hamilton--Jacobi--Bellman equationscited by 1
  31. 2024Hidden monotonicity and canonical transformations for mean field games and master equationscited by 1

Built on by

Not cited yet.