mlsanityData to 14 Oct 2026

Mitigating Gradient Pathology in PINNs through Aligned Constraint

arXiv 2605.25001 ↗First posted 24 May 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

While Physics-Informed Neural Networks (PINNs) are powerful for solving Partial Differential Equations (PDEs), their training is often paralyzed by gradient pathology. The gradients from the PDE residuals and boundary constraints oppose each other, trapping the model in local minima. Current solutions, such as adaptive weighting or hard constraints, either fail to fundamentally resolve this ill-conditioning or are limited to simple geometries. In this study, we systematically analyze the possible causes of this gradient pathology from the perspectives of loss landscapes and optimization dynamics. Based on the obtained conclusion, we propose Constraint-Aligned loss with Manifold Lifting (CAML). By reformulating all zeroth-order terms into aligned constraints, our method effectively mitigates gradient conflicts. In addition, we introduce a delay factor to help the optimizer skip the high-curvature area. Experiments demonstrate that our CAML significantly enhances numerical stability and efficiency in highly complex PINN problems. Our code is open-sourced on https://github.com/YichenLuo-0/CAML.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 51 references that are in the map. First posted on arXiv 24 May 2026.

Builds on

29 of 51 references.

References · 29

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
  3. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
  4. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  5. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  6. 2020Neural Operator: Graph Kernel Network for Partial Differential Equationscited by 734
  7. 2021Characterizing possible failure modes in physics-informed neural networkscited by 643
  8. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 634
  9. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 588
  10. 2021Physics-informed neural networks with hard constraints for inverse designcited by 280
  11. 2021Gradient-enhanced physics-informed neural networks for forward and inverse PDE problemscited by 222
  12. 2021Exact imposition of boundary conditions with distance functions in physics-informed deep neural networkscited by 137
17 more
  1. 2024Challenges in Training PINNs: A Loss Landscape Perspectivecited by 113
  2. 2018Adapting Auxiliary Losses Using Gradient Similaritycited by 100
  3. 2024PirateNets: Physics-informed Deep Learning with Residual Adaptive Networkscited by 72
  4. 2023PINNsFormer: A Transformer-Based Framework For Physics-Informed Neural Networkscited by 64
  5. 2021Meta-learning PINN loss functionscited by 51
  6. 2024ConFIG: Towards Conflict-free Training of Physics Informed Neural Networkscited by 46
  7. 2020PFNN: A Penalty-Free Neural Network Method for Solving a Class of Second-Order Boundary-Value Problems on Complex Geometriescited by 38
  8. 2024PINNACLE: PINN Adaptive ColLocation and Experimental points selectioncited by 32
  9. 2022Is L2 Physics-Informed Loss Always Suitable for Training Physics-Informed Neural Network?cited by 23
  10. 2024Dual Cone Gradient Descent for Training Physics-Informed Neural Networkscited by 21
  11. 2024Self-adaptive weights based on balanced residual decay rate for physics-informed neural networks and deep operator networkscited by 20
  12. 2023Neural Spectral Methods: Self-supervised learning in the spectral domaincited by 14
  13. 2022A Unified Hard-Constraint Framework for Solving Geometrically Complex PDEscited by 12
  14. 2024Learning a Neural Solver for Parametric PDE to Enhance Physics-Informed Methodscited by 10
  15. 2025Dual-Balancing for Physics-Informed Neural Networkscited by 5
  16. 2024Physics-Informed Neural Networks: Minimizing Residual Loss with Wide Networks and Effective Activationscited by 4
  17. 2024Beyond Derivative Pathology of PINNs: Variable Splitting Strategy with Convergence Analysiscited by 4

Built on by

Not cited yet.