# Generalization bound of globally optimal non-convex neural network training: Transportation map estimation by infinite dimensional Langevin dynamics

arXiv: https://arxiv.org/abs/2007.05824 · HTML: https://mlsanity.com/paper/2007.05824/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jul 2020.

## Built on by (most cited first, 14 of 14)

- [User-friendly introduction to PAC-Bayes bounds](https://mlsanity.com/paper/2110.11216/) (2021)
- [Stochastic Optimal Control for Diffusion Bridges in Function Spaces](https://mlsanity.com/paper/2405.20630/) (2024)
- [Fully-Connected Network on Noncompact Symmetric Space and Ridgelet Transform based on Helgason-Fourier Analysis](https://mlsanity.com/paper/2203.01631/) (2022)
- [Universality of Group Convolutional Neural Networks Based on Ridgelet Analysis on Groups](https://mlsanity.com/paper/2205.14819/) (2022)
- [Deep Networks Provably Classify Data on Curves](https://mlsanity.com/paper/2107.14324/) (2021)
- [Joint Group Invariant Functions on Data-Parameter Domain Induce Universal Neural Networks](https://mlsanity.com/paper/2310.03530/) (2023)
- [Functional Adjoint Sampler: Scalable Sampling on Infinite Dimensional Spaces](https://mlsanity.com/paper/2511.06239/) (2025)
- [Tropical Support Vector Machines: Evaluations and Extension to Function Spaces](https://mlsanity.com/paper/2101.11531/) (2021)
- [Weak Convergence of Approximate reflection coupling and its Application to Non-convex Optimization](https://mlsanity.com/paper/2205.11970/) (2022)
- [Deep Ridgelet Transform: Voice with Koopman Operator Proves Universality of Formal Deep Networks](https://mlsanity.com/paper/2310.03529/) (2023)
- [A unified Fourier slice method to derive ridgelet transform for a variety of depth-2 neural networks](https://mlsanity.com/paper/2402.15984/) (2024)
- [Uniform Generalization Bound on Time and Inverse Temperature for Gradient Descent Algorithm and its Application to Analysis of Simulated Annealing](https://mlsanity.com/paper/2205.12959/) (2022)
- [Adam-like Algorithm with Smooth Clipping Attains Global Minima: Analysis Based on Ergodicity of Functional SDEs](https://mlsanity.com/paper/2312.02182/) (2023)
- [Complex normalizing flows can almost be information K"ahler-Ricci flows](https://mlsanity.com/paper/2604.17954/) (2026)

## Abstract (arXiv)

We introduce a new theoretical framework to analyze deep learning optimization with connection to its generalization error. Existing frameworks such as mean field theory and neural tangent kernel theory for neural network optimization analysis typically require taking limit of infinite width of the network to show its global convergence. This potentially makes it difficult to directly deal with finite width network; especially in the neural tangent kernel regime, we cannot reveal favorable properties of neural networks beyond kernel methods. To realize more natural analysis, we consider a completely different approach in which we formulate the parameter training as a transportation map estimation and show its global convergence via the theory of the infinite dimensional Langevin dynamics. This enables us to analyze narrow and wide networks in a unifying manner. Moreover, we give generalization gap and excess risk bounds for the solution obtained by the dynamics. The excess risk bound achieves the so-called fast learning rate. In particular, we show an exponential convergence for a classification problem and a minimax optimal rate for a regression problem.