mlsanityData to 14 Oct 2026

Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategies

arXiv 2306.17648 ↗First posted 30 Jun 2023math.NA cs.LG cs.NAcited by 20

Not on the map. Why?

Abstract (arXiv)

We propose to enhance the training of physics-informed neural networks (PINNs). To this aim, we introduce nonlinear additive and multiplicative preconditioning strategies for the widely used L-BFGS optimizer. The nonlinear preconditioners are constructed by utilizing the Schwarz domain-decomposition framework, where the parameters of the network are decomposed in a layer-wise manner. Through a series of numerical experiments, we demonstrate that both, additive and multiplicative preconditioners significantly improve the convergence of the standard L-BFGS optimizer, while providing more accurate solutions of the underlying partial differential equations. Moreover, the additive preconditioner is inherently parallel, thus giving rise to a novel approach to model parallelism.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 33 of its 83 references that are in the map. First posted on arXiv 30 Jun 2023.

Builds on

33 of 83 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  5. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
  6. 2017DGM: A deep learning algorithm for solving partial differential equationscited by 678
  7. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 634
  8. 2017The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problemscited by 583
  9. 2021Physics-informed neural networks with hard constraints for inverse designcited by 280
  10. 2020Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanismcited by 233
  11. 2018Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysiscited by 156
  12. 2021Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equationscited by 145
21 more
  1. 2017Practical Gauss-Newton Optimisation for Deep Learningcited by 142
  2. 2021Exact imposition of boundary conditions with distance functions in physics-informed deep neural networkscited by 137
  3. 2021Parallel Physics-Informed Neural Networks via Domain Decompositioncited by 112
  4. 2020Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating a class of inverse problems for PDEscited by 102
  5. 2021When Do Extended Physics-Informed Neural Networks (XPINNs) Improve Generalization?cited by 58
  6. 2019D3M: A deep domain decomposition method for partial differential equationscited by 44
  7. 2022Augmented Physics-Informed Neural Networks (APINNs): A gating network-based soft domain decomposition methodologycited by 44
  8. 2019Robust Training and Initialization of Deep Neural Networks: An Adaptive Basis Viewpointcited by 42
  9. 2020PFNN: A Penalty-Free Neural Network Method for Solving a Class of Second-Order Boundary-Value Problems on Complex Geometriescited by 38
  10. 2018Layer-Parallel Training of Deep Residual Neural Networkscited by 35
  11. 2019Distributed physics informed neural network for data-efficient solution to partial differential equationscited by 26
  12. 2020Large-scale Neural Solvers for Partial Differential Equationscited by 17
  13. 2021Partition of unity networks: deep hp-approximationcited by 16
  14. 2020Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projectioncited by 12
  15. 2020Multilevel Minimization for Deep Residual Networkscited by 11
  16. 2021Globally Convergent Multilevel Training of Deep Residual Networkscited by 11
  17. 2023Ensemble learning for Physics Informed Neural Networks: a Gradient Boosting approachcited by 10
  18. 2021A coarse space acceleration of deep-DDMcited by 8
  19. 2023A Block-Coordinate Approach of Multi-level Optimization with an Application to Physics-Informed Neural Networkscited by 7
  20. 2023Multilevel Objective-Function-Free Optimization with an Application to Neural Networks Trainingcited by 6
  21. 2022Nonlinear Schwarz preconditioning for nonlinear optimization problems with bound constraintscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 9; 2025: 3; 2026: 4.

20 papers.

Sort:

  1. 2024From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learningcited by 73
  2. 2023Residual-based attention in physics-informed neural networkscited by 72
  3. 2024Unveiling the optimization process of Physics Informed Neural Networks: How accurate and competitive can PINNs be?cited by 31
  4. 2023An operator preconditioning perspective on training in physics-informed machine learningcited by 27
  5. 2023Machine learning and domain decomposition methods -- a surveycited by 14
  6. 2024Learning in PINNs: Phase transition, diffusion equilibrium, and generalizationcited by 13
  7. 2024Learning a Neural Solver for Parametric PDE to Enhance Physics-Informed Methodscited by 10
  8. 2024Initialization-enhanced Physics-Informed Neural Network with Domain Decomposition (IDPINN)cited by 9
  9. 2025Leveraging Operator Learning to Accelerate Convergence of the Preconditioned Conjugate Gradient Methodcited by 6
  10. 2024Two-level overlapping additive Schwarz preconditioner for training scientific machine learning applicationscited by 5
  11. 2024A Nonoverlapping Domain Decomposition Method for Extreme Learning Machines: Elliptic Problemscited by 4
  12. 2025AB-PINNs: Adaptive-Basis Physics-Informed Neural Networks for Residual-Driven Domain Decompositioncited by 4
8 more
  1. 2025Recursive Bound-Constrained AdaGrad with Applications to Multilevel and Domain Decomposition Minimizationcited by 3
  2. 2023Parallel Trust-Region Approaches in Neural Network Training: Beyond Traditional Methodscited by 2
  3. 2024Preconditioned FEM-based Neural Networks for Solving Incompressible Fluid Flows and Related Inverse Problemscited by 2
  4. 2024A Natural Primal-Dual Hybrid Gradient Method for Adversarial Neural Network Training on Solving Partial Differential Equationscited by 2
  5. 2026Multi-Preconditioned LBFGS for Training Finite-Basis PINNscited by 2
  6. 2026Sparse Random-Feature Neural Networks with Krylov-Based SVD for Singularly Perturbed ODEcited by 0
  7. 2026Smooth Piecewise Cutting for Neural Operator to Handle Discontinuities and Sharp Transitionscited by 0
  8. 2026Two-level domain-decomposition AdaGrad method for scalable training of graph neural networkscited by 0