Enhancing Stability of Physics-Informed Neural Network Training Through Saddle-Point Reformulation
Not on the map. Why?
Abstract (arXiv)
Physics-informed neural networks (PINNs) have gained prominence in recent years and are now effectively used in a number of applications. However, their performance remains unstable due to the complex landscape of the loss function. To address this issue, we reformulate PINN training as a nonconvex-strongly concave saddle-point problem. After establishing the theoretical foundation for this approach, we conduct an extensive experimental study, evaluating its effectiveness across various tasks and architectures. Our results demonstrate that the proposed method outperforms the current state-of-the-art techniques.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 39 of its 72 references that are in the map. First posted on arXiv 21 Jul 2025.
Builds on
39 of 72 references.
References · 39
- 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
- 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
- 2020Gradient Surgery for Multi-Task Learningcited by 1,256
- 2017A Survey on Multi-Task Learningcited by 840
- 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 834
- 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
- 2021Characterizing possible failure modes in physics-informed neural networkscited by 643
- 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 634
- 2019DeepXDE: A deep learning library for solving differential equationscited by 619
- 2017The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problemscited by 583
- 2018End-to-End Multi-Task Learning with Attentioncited by 534
- 2019Agnostic Federated Learningcited by 456
27 more
- 2019On Gradient Descent Ascent for Nonconvex-Concave Minimax Problemscited by 270
- 2021Gradient-enhanced physics-informed neural networks for forward and inverse PDE problemscited by 222
- 2018Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantificationcited by 199
- 2020Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Modelscited by 169
- 2019Solving a Class of Non-Convex Min-Max Games Using Iterative First Order Methodscited by 148
- 2016Relatively-Smooth Convex Optimization by First-Order Methods, and Applicationscited by 108
- 2021Multi-Objective Loss Balancing for Physics-Informed Deep Learningcited by 87
- 2023PINNacle: A Comprehensive Benchmark of Physics-Informed Neural Networks for Solving PDEscited by 87
- 2019Efficient Algorithms for Smooth Minimax Optimizationcited by 76
- 2023Residual-based attention in physics-informed neural networkscited by 72
- 2023PINNsFormer: A Transformer-Based Framework For Physics-Informed Neural Networkscited by 64
- 2020A Unified Single-loop Alternating Gradient Projection Algorithm for Nonconvex-Concave and Convex-Nonconcave Minimax Problemscited by 57
- 2020A Dual-Dimer Method for Training Physics-Constrained Neural Networks with Minimax Architecturecited by 50
- 2023Hypernetwork-based Meta-Learning for Low-Rank Physics-Informed Neural Networkscited by 44
- 2021Inverse-Dirichlet Weighting Enables Reliable Training of Physics Informed Neural Networkscited by 43
- 2019SoftAdapt: Techniques for Adaptive Loss Weighting of Neural Networks with Multi-Part Loss Functionscited by 40
- 2022Enhanced Physics-Informed Neural Networks with Augmented Lagrangian Relaxation Method (AL-PINNs)cited by 35
- 2019An accelerated inexact proximal point method for solving nonconvex-concave min-max problemscited by 30
- 2018Linear Convergence of the Primal-Dual Gradient Method for Convex-Concave Saddle Point Problems without Strong Convexitycited by 29
- 2018Local Saddle Point Optimization: A Curvature Exploitation Approachcited by 28
- 2024The Challenges of the Nonlinear Regime for Physics-Informed Neural Networkscited by 21
- 2024Dual Cone Gradient Descent for Training Physics-Informed Neural Networkscited by 21
- 2019What is Local Optimality in Nonconvex-Nonconcave Minimax Optimization?cited by 20
- 2023MultiAdam: Parameter-wise Scale-invariant Optimizer for Multiscale Training of Physics-informed Neural Networkscited by 17
- 2022A Unified Analysis of Variational Inequality Methods: Variance Reduction, Sampling, Quantization and Coordinate Descentcited by 4
- 2023Projection-Free Methods for Solving Nonconvex-Concave Saddle Point Problemscited by 1
- 2024Drago: Primal-Dual Coupled Variance Reduction for Faster Distributionally Robust Optimizationcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.