Understanding Transfer Learning via Mean-field Analysis
Not on the map. Why?
Abstract (arXiv)
We propose a novel framework for exploring generalization errors of transfer learning through the lens of differential calculus on the space of probability measures. In particular, we consider two main transfer learning scenarios, α-ERM and fine-tuning with the KL-regularized empirical risk minimization and establish generic conditions under which the generalization error and the population risk convergence rates for these scenarios are studied. Based on our theoretical results, we show the benefits of transfer learning with a one-hidden-layer neural network in the mean-field regime under some suitable integrability and regularity assumptions on the loss and activation functions.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 29 of its 45 references that are in the map. First posted on arXiv 22 Oct 2024.
Builds on
29 of 45 references.
References · 29
- 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
- 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,323
- 2014How transferable are features in deep neural networks?cited by 1,216
- 2019On Exact Computation with an Infinitely Wide Neural Netcited by 572
- 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 562
- 2018A Mean Field View of the Landscape of Two-Layers Neural Networkscited by 463
- 2018On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transportcited by 422
- 2019The generalization error of random features regression: Precise asymptotics and double descent curvecited by 410
- 2018Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layerscited by 340
- 2018Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Datacited by 260
- 2019Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networkscited by 221
- 2019Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limitcited by 187
17 more
- 2020On the Theory of Transfer Learning: The Importance of Task Diversitycited by 186
- 2023The emergence of clusters in self-attention dynamicscited by 133
- 2019Polylogarithmic width suffices for gradient descent to achieve arbitrarily small test error with shallow ReLU networkscited by 120
- 2019How Much Over-parameterization Is Sufficient to Learn Deep ReLU Networks?cited by 84
- 2020On the Value of Target Data in Transfer Learningcited by 62
- 2019Mean-Field Langevin Dynamics and Energy Landscape of Neural Networkscited by 49
- 2019Mean Field Analysis of Deep Neural Networkscited by 42
- 2020Minimax Lower Bounds for Transfer Learning with Linear and One-hidden Layer Neural Networkscited by 30
- 2020Information-theoretic analysis for transfer learningcited by 23
- 2020Particle Dual Averaging: Optimization of Mean Field Neural Networks with Global Convergence Rate Analysiscited by 21
- 2022Infinite-width limit of deep linear neural networkscited by 20
- 2019Mean-Field Neural ODEs via Relaxed Optimal Controlcited by 16
- 2020Mathematical Models of Overparameterized Neural Networkscited by 15
- 2021Characterizing and Understanding the Generalization Error of Transfer Learning with Gibbs Algorithmcited by 14
- 2019Over Parameterized Two-level Neural Networks Can Learn Near Optimal Feature Representationscited by 7
- 2023Mean-field Analysis of Generalization Errorscited by 5
- 2024Generalization Error of Graph Neural Networks in the Mean-field Regimecited by 4
Built on by
Citing papers per year: 2025: 1.
1 paper.