mlsanityData to 7 Oct 2026

Multilevel Objective-Function-Free Optimization with an Application to Neural Networks Training

arXiv 2302.07049 ↗First posted 14 Feb 2023math.OC cs.AI cs.CCcited by 6

Not on the map. Why?

Abstract (arXiv)

A class of multi-level algorithms for unconstrained nonlinear optimization is presented which does not require the evaluation of the objective function. The class contains the momentum-less AdaGrad method as a particular (single-level) instance. The choice of avoiding the evaluation of the objective function is intended to make the algorithms of the class less sensitive to noise, while the multi-level feature aims at reducing their computational cost. The evaluation complexity of these algorithms is analyzed and their behaviour in the presence of noise is then illustrated in the context of training deep neural networks for supervised learning applications.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 14 of its 61 references that are in the map. First posted on arXiv 14 Feb 2023.

Builds on

14 of 61 references.

References · 14

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  3. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  4. 2019On the Convergence of Adam and Beyondcited by 834
  5. 2017Stable Architectures for Deep Neural Networkscited by 340
  6. 2018On the Convergence of Adaptive Gradient Methods for Nonconvex Optimizationcited by 91
  7. 2017Multi-level Residual Networks from Dynamical Systems Viewcited by 59
  8. 2020Continuous-in-Depth Neural Networkscited by 40
  9. 2020Multilevel Minimization for Deep Residual Networkscited by 11
  10. 2021Globally Convergent Multilevel Training of Deep Residual Networkscited by 11
  11. 2019Multilevel Initialization for Layer-Parallel Deep Neural Network Trainingcited by 7
  12. 2014Distributed Block Coordinate Descent for Minimizing Partially Separable Functionscited by 3
2 more
  1. 2022Complexity and performance for two classes of noise-tolerant first-order algorithmscited by 3
  2. 2022Convergence properties of an Objective-Function-Free Optimization regularization algorithm, including an O(ε-3/2) complexity boundcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 2; 2026: 1.

6 papers.

  1. 2023Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategiescited by 20
  2. 2025Leveraging Operator Learning to Accelerate Convergence of the Preconditioned Conjugate Gradient Methodcited by 6
  3. 2024Two-level overlapping additive Schwarz preconditioner for training scientific machine learning applicationscited by 5
  4. 2025Recursive Bound-Constrained AdaGrad with Applications to Multilevel and Domain Decomposition Minimizationcited by 3
  5. 2023Parallel Trust-Region Approaches in Neural Network Training: Beyond Traditional Methodscited by 2
  6. 2026Two-level domain-decomposition AdaGrad method for scalable training of graph neural networkscited by 0