mlsanityData to 14 Oct 2026

A Block-Coordinate Approach of Multi-level Optimization with an Application to Physics-Informed Neural Networks

arXiv 2305.14477 ↗First posted 23 May 2023cs.LG math.OCcited by 7

Not on the map. Why?

Abstract (arXiv)

Multi-level methods are widely used for the solution of large-scale problems, because of their computational advantages and exploitation of the complementarity between the involved sub-problems. After a re-interpretation of multi-level methods from a block-coordinate point of view, we propose a multi-level algorithm for the solution of nonlinear optimization problems and analyze its evaluation complexity. We apply it to the solution of partial differential equations using physics-informed neural networks (PINNs) and show on a few test problems that the approach results in better solutions and significant computational savings

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 24 of its 51 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

24 of 51 references.

References · 24

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,200
  3. 2018On the Spectral Bias of Neural Networkscited by 1,341
  4. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,150
  5. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 634
  6. 2017The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problemscited by 583
  7. 2021Physics-informed neural networks (PINNs) for fluid mechanics: A reviewcited by 473
  8. 2019Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networkscited by 368
  9. 2017Stable Architectures for Deep Neural Networkscited by 341
  10. 2020On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networkscited by 284
  11. 2015Coordinate Descent Algorithmscited by 219
  12. 2019The Convergence Rate of Neural Networks for Learned Functions of Different Frequenciescited by 135
12 more
  1. 2016Hierarchical Multiscale Recurrent Neural Networkscited by 104
  2. 2020Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating a class of inverse problems for PDEscited by 102
  3. 2020Multi-scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann Equation in Complex Domainscited by 79
  4. 2019A Multigrid Method for Efficiently Training Video Modelscited by 52
  5. 2021Globally Convergent Multilevel Training of Deep Residual Networkscited by 11
  6. 2021Subspace Decomposition based DNN algorithm for elliptic type multi-scale PDEscited by 10
  7. 2022The cost of nonconvexity in deterministic nonsmooth optimizationcited by 9
  8. 2022On the Complexity of Deterministic Nonsmooth and Nonconvex Optimizationcited by 6
  9. 2022Momentum Diminishes the Effect of Spectral Bias in Physics-Informed Neural Networkscited by 4
  10. 2021On the Globalization of ASPIN Employing Trust-Region Control Strategies -- Convergence Analysis and Numerical Examplescited by 3
  11. 2021Training of deep residual networks with stochastic MG/OPTcited by 2
  12. 2020On complexity and convergence of high-order coordinate descent algorithms for smooth nonconvex box-constrained minimizationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 2; 2026: 2.

7 papers.

  1. 2023Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategiescited by 20
  2. 2024A Nonoverlapping Domain Decomposition Method for Extreme Learning Machines: Elliptic Problemscited by 4
  3. 2025Recursive Bound-Constrained AdaGrad with Applications to Multilevel and Domain Decomposition Minimizationcited by 3
  4. 2024Two-level deep domain decomposition methodcited by 0
  5. 2025A multilevel approach to accelerate the training of Transformerscited by 0
  6. 2026A block-coordinate descent framework for non-convex composite optimization. Application to sparse precision matrix estimationcited by 0
  7. 2026Stochastic convergence of parallel asynchronous adaptive first-order methodscited by 0