# An adaptive augmented Lagrangian method for training physics and equality constrained artificial neural networks

arXiv: https://arxiv.org/abs/2306.04904 · HTML: https://mlsanity.com/paper/2306.04904/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 36 references that are in the map.
- First posted on arXiv 8 Jun 2023.

## Builds on (most cited first, 15 of 15)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Physics-informed neural networks with hard constraints for inverse design](https://mlsanity.com/paper/2102.04626/) (2021)
- [Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data](https://mlsanity.com/paper/1901.06314/) (2019)
- [A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks](https://mlsanity.com/paper/2207.10289/) (2022)
- [Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism](https://mlsanity.com/paper/2009.04544/) (2020)
- [Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10566/) (2017)
- [A Dual-Dimer Method for Training Physics-Constrained Neural Networks with Minimax Architecture](https://mlsanity.com/paper/2005.00615/) (2020)
- [Physics and Equality Constrained Artificial Neural Networks: Application to Forward and Inverse Problems with Multi-fidelity Data Fusion](https://mlsanity.com/paper/2109.14860/) (2021)
- [Training neural networks under physical constraints using a stochastic augmented Lagrangian approach](https://mlsanity.com/paper/2009.07330/) (2020)
- [Critical Investigation of Failure Modes in Physics-informed Neural Networks](https://mlsanity.com/paper/2206.09961/) (2022)

## Built on by (most cited first, 16 of 16)

- [From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learning](https://mlsanity.com/paper/2410.13228/) (2024)
- [Residual-based attention in physics-informed neural networks](https://mlsanity.com/paper/2307.00379/) (2023)
- [Learning in PINNs: Phase transition, diffusion equilibrium, and generalization](https://mlsanity.com/paper/2403.18494/) (2024)
- [Geometry-Informed Neural Networks](https://mlsanity.com/paper/2402.14009/) (2024)
- [A Variational Framework for Residual-Based Adaptivity in Neural PDE Solvers and Operator Learning](https://mlsanity.com/paper/2509.14198/) (2025)
- [Self-adaptive weighting and sampling for physics-informed neural networks](https://mlsanity.com/paper/2511.05452/) (2025)
- [A Generalized Schwarz-type Non-overlapping Domain Decomposition Method using Physics-constrained Neural Networks](https://mlsanity.com/paper/2307.12435/) (2023)
- [Non-overlapping, Schwarz-type Domain Decomposition Method for Physics and Equality Constrained Artificial Neural Networks](https://mlsanity.com/paper/2409.13644/) (2024)
- [Conditionally adaptive augmented Lagrangian method for physics-informed learning of forward and inverse problems](https://mlsanity.com/paper/2508.15695/) (2025)
- [Common pitfalls to avoid while using multiobjective optimization in machine learning](https://mlsanity.com/paper/2405.01480/) (2024)
- [Error Bounds for Physics-Informed Neural Networks in Fokker-Planck PDEs](https://mlsanity.com/paper/2410.22371/) (2024)
- [Diverse Topology Optimization using Modulated Neural Fields](https://mlsanity.com/paper/2502.13174/) (2025)
- [Large-scale portfolio optimization with variational neural annealing](https://mlsanity.com/paper/2507.07159/) (2025)
- [Fundamentals of Regression](https://mlsanity.com/paper/2512.01920/) (2025)
- [GLUE: Coordinating Pre-Trained Generative Models for System-Level Design](https://mlsanity.com/paper/2512.19469/) (2025)
- [Only Project Once: Projection-Adaptive Loss for Exact Constraint Satisfaction](https://mlsanity.com/paper/2610.04572/) (2026)

## Abstract (arXiv)

Physics and equality constrained artificial neural networks (PECANN) are grounded in methods of constrained optimization to properly constrain the solution of partial differential equations (PDEs) with their boundary and initial conditions and any high-fidelity data that may be available. To this end, adoption of the augmented Lagrangian method within the PECANN framework is paramount for learning the solution of PDEs without manually balancing the individual loss terms in the objective function used for determining the parameters of the neural network. Generally speaking, ALM combines the merits of the penalty and Lagrange multiplier methods while avoiding the ill conditioning and convergence issues associated singly with these methods . In the present work, we apply our PECANN framework to solve forward and inverse problems that have an expanded and diverse set of constraints. We show that ALM with its conventional formulation to update its penalty parameter and Lagrange multipliers stalls for such challenging problems. To address this issue, we propose an adaptive ALM in which each constraint is assigned a unique penalty parameter that evolve adaptively according to a rule inspired by the adaptive subgradient method. Additionally, we revise our PECANN formulation for improved computational efficiency and savings which allows for mini-batch training. We demonstrate the efficacy of our proposed approach by solving several forward and PDE-constrained inverse problems with noisy data, including simulation of incompressible fluid flows with a primitive-variables formulation of the Navier-Stokes equations up to a Reynolds number of 1000.