# A Block-Coordinate Approach of Multi-level Optimization with an Application to Physics-Informed Neural Networks

arXiv: https://arxiv.org/abs/2305.14477 · HTML: https://mlsanity.com/paper/2305.14477/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 51 references that are in the map.
- First posted on arXiv 23 May 2023.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks](https://mlsanity.com/paper/1901.06523/) (2019)
- [Stable Architectures for Deep Neural Networks](https://mlsanity.com/paper/1705.03341/) (2017)
- [On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks](https://mlsanity.com/paper/2012.10047/) (2020)
- [Coordinate Descent Algorithms](https://mlsanity.com/paper/1502.04759/) (2015)
- [The Convergence Rate of Neural Networks for Learned Functions of Different Frequencies](https://mlsanity.com/paper/1906.00425/) (2019)
- [Hierarchical Multiscale Recurrent Neural Networks](https://mlsanity.com/paper/1609.01704/) (2016)
- [Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating a class of inverse problems for PDEs](https://mlsanity.com/paper/2007.01138/) (2020)
- [Multi-scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann Equation in Complex Domains](https://mlsanity.com/paper/2007.11207/) (2020)
- [A Multigrid Method for Efficiently Training Video Models](https://mlsanity.com/paper/1912.00998/) (2019)
- [Globally Convergent Multilevel Training of Deep Residual Networks](https://mlsanity.com/paper/2107.07572/) (2021)
- [Subspace Decomposition based DNN algorithm for elliptic type multi-scale PDEs](https://mlsanity.com/paper/2112.06660/) (2021)
- [The cost of nonconvexity in deterministic nonsmooth optimization](https://mlsanity.com/paper/2210.00652/) (2022)
- [On the Complexity of Deterministic Nonsmooth and Nonconvex Optimization](https://mlsanity.com/paper/2209.12463/) (2022)
- [Momentum Diminishes the Effect of Spectral Bias in Physics-Informed Neural Networks](https://mlsanity.com/paper/2206.14862/) (2022)
- [On the Globalization of ASPIN Employing Trust-Region Control Strategies -- Convergence Analysis and Numerical Examples](https://mlsanity.com/paper/2104.05672/) (2021)
- [Training of deep residual networks with stochastic MG/OPT](https://mlsanity.com/paper/2108.04052/) (2021)
- [On complexity and convergence of high-order coordinate descent algorithms for smooth nonconvex box-constrained minimization](https://mlsanity.com/paper/2009.01811/) (2020)

## Built on by (most cited first, 7 of 7)

- [Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategies](https://mlsanity.com/paper/2306.17648/) (2023)
- [A Nonoverlapping Domain Decomposition Method for Extreme Learning Machines: Elliptic Problems](https://mlsanity.com/paper/2406.15959/) (2024)
- [Recursive Bound-Constrained AdaGrad with Applications to Multilevel and Domain Decomposition Minimization](https://mlsanity.com/paper/2507.11513/) (2025)
- [Two-level deep domain decomposition method](https://mlsanity.com/paper/2408.12198/) (2024)
- [A multilevel approach to accelerate the training of Transformers](https://mlsanity.com/paper/2504.18590/) (2025)
- [A block-coordinate descent framework for non-convex composite optimization. Application to sparse precision matrix estimation](https://mlsanity.com/paper/2601.21467/) (2026)
- [Stochastic convergence of parallel asynchronous adaptive first-order methods](https://mlsanity.com/paper/2606.01787/) (2026)

## Abstract (arXiv)

Multi-level methods are widely used for the solution of large-scale problems, because of their computational advantages and exploitation of the complementarity between the involved sub-problems. After a re-interpretation of multi-level methods from a block-coordinate point of view, we propose a multi-level algorithm for the solution of nonlinear optimization problems and analyze its evaluation complexity. We apply it to the solution of partial differential equations using physics-informed neural networks (PINNs) and show on a few test problems that the approach results in better solutions and significant computational savings