# Enhanced Physics-Informed Neural Networks with Augmented Lagrangian Relaxation Method (AL-PINNs)

arXiv: https://arxiv.org/abs/2205.01059 · HTML: https://mlsanity.com/paper/2205.01059/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 49 references that are in the map.
- First posted on arXiv 29 Apr 2022.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks](https://mlsanity.com/paper/2012.10047/) (2020)
- [Physics-informed neural networks with hard constraints for inverse design](https://mlsanity.com/paper/2102.04626/) (2021)
- [Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism](https://mlsanity.com/paper/2009.04544/) (2020)
- [Exact imposition of boundary conditions with distance functions in physics-informed deep neural networks](https://mlsanity.com/paper/2104.08426/) (2021)
- [Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networks](https://mlsanity.com/paper/2007.04542/) (2020)
- [A unified deep artificial neural network approach to partial differential equations in complex geometries](https://mlsanity.com/paper/1711.06464/) (2017)
- [Multi-Objective Loss Balancing for Physics-Informed Deep Learning](https://mlsanity.com/paper/2110.09813/) (2021)
- [Lagrangian Duality for Constrained Deep Learning](https://mlsanity.com/paper/2001.09394/) (2020)
- [Learning in Sinusoidal Spaces with Physics-Informed Neural Networks](https://mlsanity.com/paper/2109.09338/) (2021)
- [Deep Kronecker neural networks: A general framework for neural networks with adaptive activation functions](https://mlsanity.com/paper/2105.09513/) (2021)
- [DPM: A Novel Training Method for Physics-Informed Neural Networks in Extrapolation](https://mlsanity.com/paper/2012.02681/) (2020)
- [Imposing Hard Constraints on Deep Networks: Promises and Limitations](https://mlsanity.com/paper/1706.02025/) (2017)
- [Physics and Equality Constrained Artificial Neural Networks: Application to Forward and Inverse Problems with Multi-fidelity Data Fusion](https://mlsanity.com/paper/2109.14860/) (2021)
- [Data vs. Physics: The Apparent Pareto Front of Physics-Informed Neural Networks](https://mlsanity.com/paper/2105.00862/) (2021)
- [Trend to Equilibrium for the Kinetic Fokker-Planck Equation via the Neural Network Approach](https://mlsanity.com/paper/1911.09843/) (2019)
- [Deep Neural Network Approach to Forward-Inverse Problems](https://mlsanity.com/paper/1907.12925/) (2019)
- [An augmented Lagrangian deep learning method for variational problems with essential boundary conditions](https://mlsanity.com/paper/2106.14348/) (2021)
- [Deep Ritz revisited](https://mlsanity.com/paper/1912.03937/) (2019)
- [Lagrangian dual framework for conservative neural network solutions of kinetic equations](https://mlsanity.com/paper/2106.12147/) (2021)

## Built on by (most cited first, 30 of 35)

- [From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learning](https://mlsanity.com/paper/2410.13228/) (2024)
- [Residual-based attention in physics-informed neural networks](https://mlsanity.com/paper/2307.00379/) (2023)
- [ConFIG: Towards Conflict-free Training of Physics Informed Neural Networks](https://mlsanity.com/paper/2408.11104/) (2024)
- [Dual Cone Gradient Descent for Training Physics-Informed Neural Networks](https://mlsanity.com/paper/2409.18426/) (2024)
- [Self-adaptive weights based on balanced residual decay rate for physics-informed neural networks and deep operator networks](https://mlsanity.com/paper/2407.01613/) (2024)
- [Learning in PINNs: Phase transition, diffusion equilibrium, and generalization](https://mlsanity.com/paper/2403.18494/) (2024)
- [Geometry-Informed Neural Networks](https://mlsanity.com/paper/2402.14009/) (2024)
- [Complex Physics-Informed Neural Network](https://mlsanity.com/paper/2502.04917/) (2025)
- [A Variational Framework for Residual-Based Adaptivity in Neural PDE Solvers and Operator Learning](https://mlsanity.com/paper/2509.14198/) (2025)
- [Self-adaptive weighting and sampling for physics-informed neural networks](https://mlsanity.com/paper/2511.05452/) (2025)
- [Discovering Physics-Informed Neural Networks Model for Solving Partial Differential Equations through Evolutionary Computation](https://mlsanity.com/paper/2405.11208/) (2024)
- [Robust Weight Initialization for Tanh Neural Networks with Fixed Point Analysis](https://mlsanity.com/paper/2410.02242/) (2024)
- [Achieving Constraints in Neural Networks: A Stochastic Augmented Lagrangian Approach](https://mlsanity.com/paper/2310.16647/) (2023)
- [Physics-Guided Neural Networks for Intraventricular Vector Flow Mapping](https://mlsanity.com/paper/2403.13040/) (2024)
- [Constrained or Unconstrained? Neural-Network-Based Equation Discovery from Data](https://mlsanity.com/paper/2406.02581/) (2024)
- [WANCO: Weak Adversarial Networks for Constrained Optimization problems](https://mlsanity.com/paper/2407.03647/) (2024)
- [Incorporating Domain Knowledge in Deep Neural Networks for Discrete Choice Models](https://mlsanity.com/paper/2306.00016/) (2023)
- [Common pitfalls to avoid while using multiobjective optimization in machine learning](https://mlsanity.com/paper/2405.01480/) (2024)
- [General-Kindred Physics-Informed Neural Network to the Solutions of Singularly Perturbed Differential Equations](https://mlsanity.com/paper/2408.14734/) (2024)
- [Boundary condition enforcement with PINNs: a comparative study and verification on 3D geometries](https://mlsanity.com/paper/2512.14941/) (2025)
- [Physics-Informed Deep B-Spline Networks](https://mlsanity.com/paper/2503.16777/) (2025)
- [Physics-informed solution reconstruction in elasticity and heat transfer using the explicit constraint force method](https://mlsanity.com/paper/2505.04875/) (2025)
- [Enhancing Stability of Physics-Informed Neural Network Training Through Saddle-Point Reformulation](https://mlsanity.com/paper/2507.16008/) (2025)
- [Visualizing the loss landscapes of physics-informed neural networks](https://mlsanity.com/paper/2602.05849/) (2026)
- [Physics-Guided Dimension Reduction for Simulation-Free Operator Learning of Stiff Differential-Algebraic Systems](https://mlsanity.com/paper/2604.19930/) (2026)
- [AdamFLIP: Adaptive Momentum Feedback Linearization Optimization for Hard Constrained PINN Training](https://mlsanity.com/paper/2605.08408/) (2026)
- [Physics-Informed Neural Networks for an optimal counterdiabatic quantum computation](https://mlsanity.com/paper/2309.04434/) (2023)
- [A Two-Stage Training Method for Modeling Constrained Systems With Neural Networks](https://mlsanity.com/paper/2403.02730/) (2024)
- [Taylor-Model Physics-Informed Neural Networks (PINNs) for Ordinary Differential Equations](https://mlsanity.com/paper/2507.03860/) (2025)
- [Fundamentals of Regression](https://mlsanity.com/paper/2512.01920/) (2025)

## Abstract (arXiv)

Physics-Informed Neural Networks (PINNs) have become a prominent application of deep learning in scientific computation, as they are powerful approximators of solutions to nonlinear partial differential equations (PDEs). There have been numerous attempts to facilitate the training process of PINNs by adjusting the weight of each component of the loss function, called adaptive loss-balancing algorithms. In this paper, we propose an Augmented Lagrangian relaxation method for PINNs (AL-PINNs). We treat the initial and boundary conditions as constraints for the optimization problem of the PDE residual. By employing Augmented Lagrangian relaxation, the constrained optimization problem becomes a sequential max-min problem so that the learnable parameters λ adaptively balance each loss component. Our theoretical analysis reveals that the sequence of minimizers of the proposed loss functions converges to an actual solution for the Helmholtz, viscous Burgers, and Klein--Gordon equations. We demonstrate through various numerical experiments that AL-PINNs yield a much smaller relative error compared with that of state-of-the-art adaptive loss-balancing algorithms.